Not seeing files in folder/requesting read permissions

[Draque Thompson]

Windows users of my application are having a problem seeing files in a folder with the default JFileChooser. Specifically when they browse to the C:\Windows\Fonts folder, it appears empty. When browsing to other folders,

I suspect this is some sort of invisible permissions issue, but I am having trouble figuring out how to nail this down or request permissions on the folder to allow its contents to be seen. Is this something that anyone else here has dealt with? The code I'm using is all very standard, but I'm including it below just for completeness. I am using Java 17 at this point.

    JFileChooser chooser = new JFileChooser();
    FileNameExtensionFilter filter
        = new FileNameExtensionFilter("Font Files", "ttf", "otf", "ttc", "dfont");

    chooser.setDialogTitle("Select Font File");
    chooser.setFileFilter(filter);
    chooser.setCurrentDirectory(core.getWorkingDirectory());
    chooser.setApproveButtonText("Open");

    if (chooser.showOpenDialog(this) == JFileChooser.APPROVE_OPTION) {
        < . . . >
    }

    < . . . >

And I hope everyone is having a good holiday! Writing this from bus station as I wait for my ride back to family.

[Rob Spoor]

This is an interesting case. I tried it myself, and both the "Fonts" and "All Files" filters show nothing at all. I thought it was a permissions issue, but that doesn't appear to be the case, as the following does yield results:

new File("C:/Windows/Fonts").listFiles();

Now I know that a JFileChooser doesn't use File directly, it uses a FileSystemView. So I tried the following:

File fonts = new File("C:/Windows/Fonts");
FileSystemView fsv = FileSystemView.getFileSystemView();
fsv.getFiles(fonts, false); // File[0] { }
fsv.getFiles(fonts, true); // File[0] { }

So it's the FileSystemView that's filtering out our files. To be continued...

[Rob Spoor]

Now, I know FileSystemView a bit, from my brief attempts to write a PathChooser (I gave up because it's actually a lot of work, and I didn't see the point anymore).

FileSystemView works not with just File, but also with an internal class called ShellFolder. The conversion can fail, so I tried that using one of the other methods that uses ShellFolder:

fsv.getSystemIcon(fonts); // Folder
fsv.getSystemDisplayName(fonts); // "Fonts"

So that conversion works. That means the culprit must be inside the getFiles method itself. That's doing quite some stuff, so let's try to mimic that. Warning: I had to use reflection because JShell doesn't allow me to access ShellFolder:

Method method = FileSystemView.class.getDeclaredMethod("getShellFolder", File.class);
method.setAccessible(true);

method.invoke(fsv, fonts); // C:\Windows\Fonts
method.invoke(fsv, fonts).getClass(); // class sun.awt.shell.Win32ShellFolder2

Good news, I can get a reference to the ShellFolder instance. Let's continue with doing what getFiles does. Unfortunately, no access to ShellFolder means more reflection...

var shellFolder = method.invoke(fsv, fonts);
method = shellFolder.getClass().getDeclaredMethod("listFiles", boolean.class);
method.setAccessible(true);
method.invoke(shellFolder, true); // ShellFolder[0] { }

So it's the ShellFolder itself that's not listing our files...

[Rob Spoor]

We're going to have to use the source now: https://github.com/openjdk/jdk/blob/master/src/java.desktop/windows/classes/sun/awt/shell/Win32ShellFolder2.java. listFiles returns an empty array, so the first culprit is line 767. Let's check that out:

method = shellFolder.getClass().getDeclaredMethod("isLink");
method.setAccessible(true);
method.invoke(shellFolder); // false

So it's not a link; that means there are some possibilities:
1) the thread is interrupted (line 808/809 or line 816)
2) the iteration doesn't find any files (line 798 is not called)
3) line 814 returns an empty array

I did some hacking in JShell like above, and I noticed that:
* The call to getEnumObjects does not return 0.
* The first call to getNextChild does not return 0.
* The first call to getAttributes0 does return 0.

Now getAttributes0 is a native method, so we have to look at native code. Thanks to a commit message I could relatively easily find the native implementation: https://github.com/openjdk/jdk/blob/master/src/java.desktop/windows/native/libawt/windows/ShellFolder2.cpp#L564. That leads me to https://docs.microsoft.com/en-us/windows/win32/api/shobjidl_core/nn-shobjidl_core-ishellfolder and https://docs.microsoft.com/en-us/windows/win32/api/shobjidl_core/nf-shobjidl_core-ishellfolder-getattributesof.

That's as far as I'm willing to go, so if you want to know exactly why files are filtered out you'll have to do some native coding of your own.

[Paul Clapham]

Makes me wonder why it's so complicated and why it's different.

However... there is a way to find all of the available fonts using standard Java code. I know because I have done it myself, my code is not where I am, but I just searched the web and found you start from the GraphicsEnvironment class.

[Draque Thompson]

Thank you for the replies there! I have done some digging of my own and found that the contents Fonts folder is actually completely virtual. Explorer itself knows to populate it with values from elsewhere. All fonts are stored in individual folders with esoteric names within c:/Windows/WinSxS.

So given that the fonts folder is virtual... anyone know a good way to read that manually? I have been getting complaints from my users, and would really like to be able to integrate this better, especially for folks who know enough to go to a specific folder to pick out a font.

Also, @Paul Clapham I've gone down that path already, and unfortunately need to do things this way. Java does not load ligatures properly for all OSes and font types unless you load directly from the binary file, which impacts my users in particular. I need to be able to present them with a standard UI.

[Rob Spoor]

Paul Clapham wrote:Makes me wonder why it's so complicated and why it's different.

Probably because of virtual folders (like Fonts, but that doesn't work apparently). For instance, "Desktop" isn't a real folder, but it does show up in file choosers.

[Paul Clapham]

Draque Thompson wrote:Also, @Paul Clapham I've gone down that path already, and unfortunately need to do things this way. Java does not load ligatures properly for all OSes and font types unless you load directly from the binary file, which impacts my users in particular. I need to be able to present them with a standard UI.

This seems like something the Java maintainers ought to be able to fix. But in practice that ain't gonna happen, so yes, unfortunately it's up to you.

[Tim Holloway]

Paul Clapham wrote:Makes me wonder why it's so complicated and why it's different.

Because fonts are not accessed as files in normal usage. They are provided as part of the font component of the graphics subsystem which uses a best-fit finder not a file finder.

That said, I seem to recall that font files had an icon in Windows and you used to be able to install fonts via drag-and-drop. But tighter security in more recent versions of Windows has likely made access to the font files administrator-only.  As I recall, for a while fonts were being infected with malware.

As far as ligatures go, I know of absolutely no reason why when you present a ligature Unicode code point that Java would not render it as a ligature. Bear in mind, however, that if you write per-character instead of per-string, it also is under no obligation to find and replace character pairs with ligatures. I have doubts about it doing so per-string for that matter. And as far as that goes, I'm doubtful that raw Windows text services would either. That's usually handled by applications such as MS-Word.

Apparently the OpenType font spec allows for ligature substitution for fonts that define it and Java has support for that. Times New Roman is one specifically mentioned. Definitely don't expect it on monospaced fonts, though.

[Draque Thompson]

This seems like something the Java maintainers ought to be able to fix. But in practice that ain't gonna happen, so yes, unfortunately it's up to you.

This times infinity. I have filed multiple bug reports, and there's actually an old post here where I specify code which specifically circumvents this exact problem with seteps to reproduce and correct... but the tickets are silently closed.

Solution if you're interested: https://coderanch.com/t/694808/java/Font-Ligatures-Swing-Display-Objects

As far as ligatures go, I know of absolutely no reason why when you present a ligature Unicode code point that Java would not render it as a ligature.

I'm not sure what the underlying cause is, but the workaround is linked above. When I load it in any way other than pulling the binary directly and manually turning the ligatures on, it fails to load them properly in many fonts. This is a bug in Java that I've been reporting without avail to Oracle for years, unfortunately.

Does anyone happen to know how to request the virtual file locations from Windows? I can make a workaround window that searches for the actual files and presents users with a custom menu... but that would be sloppy.

[Tim Holloway]

Draque Thompson wrote:
Does anyone happen to know how to request the virtual file locations from Windows? I can make a workaround window that searches for the actual files and presents users with a custom menu... but that would be sloppy.

Do you mean the locations of all installed fonts? I'm pretty sure there are published rules for directories used by Windows. Unix/Linux is much messier, since there were probably 4 different font managers in effect at one point, and they managed everything from device-dependent bitmap fonts to Adobe Type 1 to TrueType and probably others I don't recall. And on top of that, the font directories were configured, not hard-coded in the OS.

[Draque Thompson]

Do you mean the locations of all installed fonts? I'm pretty sure there are published rules for directories used by Windows.

Blorf. I think I'm going to have to end up making a special case for this that detects when the windows/fonts folder is hit and manually populates the view of Windows/Fonts based on this X(. Anyhoo, it's only a problem for Windows, but it's a problem that I unfortunately need to address before my next release.

Once I have a solution, I'll post it here so that anyone finding this thread won't be left in the cold if they have the same problem.

[Tim Holloway]

There are 2 ways you can determine fonts. One is to use the font enumeration method of the graphics subsystem.

That's the preferred one, since it shows what fonts are actually available for use by graphics services. But it doesn't list file locations.

The other way is fairly simple on Windows, since apparently unlike so many other Windows resources, the C:\Windows\Fonts* directory has been the sole authority for fonts registered with the graphics subsystem (as opposed to imported directly by applications like, maybe Adobe).

However, there are "gotchas". First, when you enumerate font files, the file names and font family names are not guaranteed identical. You'd actually have to look inside the font to get the authoritative name.

Secondly, of course, any app enumerating the font files would have to have directory-read rights on C:\Windows\Fonts. Probably on the individual files themselves. Definitely if you want to examine their insides.

Finally, the FileSystemView has the option to return or obscure files with the "hidden" attribute set. In the Unix world, a file is "hidden" if its name begins with a dot (".hideme"). But if memory serves, Windows has an actual directory attribute to hide files. So for best results, it's desirable to make sure that the FileSystemView has been configured to return hidden files as well as non-hidden files.

===
* This is, of course, assumes that Windows resides on the C drive, as is most often the case.

[Draque Thompson]

Just as a followup to this one, I did end up coming up with a solution, even if it's suboptimal. The solution is specifically for Windows, obviously.

    public static List<File> getWindowsFonts() {
        return searchForFonts(new File(System.getenv("windir") + "/WinSxS"));
    }

    /**
     * Recursively searches target directory for fonts
     * @param targetDirectory
     * @return 
     */
    public static List<File> searchForFonts(File targetDirectory) {
        var foundFonts = new ArrayList<File>();
        
        searchForFonts(targetDirectory, foundFonts);
        
        return foundFonts;
    }
    
    /**
     * Populates passed list via side effect to more easily allow for recursion
     * @param target
     * @param foundFonts 
     */
    private static void searchForFonts(File target, List<File> foundFonts) {
        if (target.exists()) {
            FileFilter filter = (File file1) -> {
                java.lang.String name1 = file1.getName();
                return (file1.isDirectory() && name1.toLowerCase().contains("font")) 
                        || name1.toLowerCase().endsWith("ttf") 
                        || name1.toLowerCase().endsWith("otf") 
                        || name1.toLowerCase().endsWith("ttc") 
                        || name1.toLowerCase().endsWith("dfont");
            };

            for (File search : target.listFiles(filter)) {
                if (search.isDirectory()) {
                    searchForFonts(search, foundFonts);
                } else {
                    foundFonts.add(search);
                }
            }
        }
    }

[Paul Clapham]

Looks like good work!

Since you mentioned Java 17 in a different post: I have been updating my code to use "new" features of Java such as NIO for files. (In this case, "not-so-new".)

I found that if you have a Path for a directory, such as what you could get in line 2 of your posted code, then there is a method Files.list(Path) which returns a Stream<Path> which will contain all of the entries in that directory. You can then use all of the normal Stream methods such as filter() and forEach().

A couple of lines from my revised version:

  try (Stream<Path> list = Files.list(path)) {
    list.filter(p -> p.toString().toLowerCase().endsWith(TEXT_SUFFIX) && Files.isRegularFile(p))
         .sorted()

[Rob Spoor]

I'd use Files.walk instead of Files.list because it supports recursive listing files:

Stream<Path> fontFiles = Files.walk(baseDir)
        .filter(Files::isRegularFile)
        .filter(p -> hasFontExtension(p.getFileName().toString()))
        ;

[Paul Clapham]

I would use Files.walk too if I wanted to search a folder recursively.

[Rob Spoor]

That's what Draque was doing, that's why I suggested it.