Mulling Things Over


Problems with symlinks and portable Lisp code

Lately, I have been contributing work to ASDF-INSTALL in an attempt to make it more configurable (especially on Windows), and to make it handle multiple defsystems correctly. This was motivated in part by dissatisfaction with the recommended techniques for dealing with Windows’ lack of symbolic links in ASDF and ASDF-INSTALL. It should, and can, be simpler: As simple as using a regular file containing a pathname to the target in lieu of a bona fide symbolic link file. Teaching ASDF to do this easy; teaching ASDF-INSTALL is not really possible right now (except in the unstable version, sort of).

Another motivation was seeing people often recommend ASDF-INSTALL to Lisp newcomers on comp.lang.lisp, only to have the new visitor return shortly thereafter with problems (usually but not always on Windows), and then have people disrecommend ASDF-INSTALL and describe alternate approaches. Which is a pity because of the wasted time, and because ASDF-INSTALL is really convenient when it does work. So it should, well, just work!

I was pleased to note the presence of a unit test suite in ASDF-INSTALL. As I started making more complex changes, I really wanted to leverage that, but I found it unreliable. The main problem is setup of scratch directory trees for the unit tests to operate on. Or more specifically, the removal of such trees. As per the usual recommendations for using ASDF, these trees often contain symbolic links to system files. Do you know how hard it is for a Common Lisp program to portably delete a directory tree that contains symbolic links with targets internal to the tree? Infernally hard! I turned to CL-FAD‘s DELETE-DIRECTORY-AND-FILES, apparently modeled on Allegro‘s function of the same name, but it can be easily confounded by symbolic links. The difficulties are legion…

There’s no portable way to delete a directory. Some Lisps allow DELETE-FILE to delete a directory; others (e.g. CLISP) take a stricter interpretation and disallow this. So immediately you need a non-standard function, typically DELETE-DIRECTORY in some implementation-specific package. (CL-FAD handles this issue internally.)

Next, whatever the means of deleting a directory, it often can’t delete a non-empty directory. When it can’t, the fun really begins. In order to make a directory empty, you have to be able to find everything in it, and delete it. This is what CL-FAD:DELETE-DIRECTORY-AND-FILES usually tries to do. But in the presence of symlinks, this gets ugly.

Finding everything in a directory can be surprisingly hard. DIRECTORY is specified to return truenames. Most Lisps interpret the truename of a symbolic link to be the ultimate target of the link. This is quite reasonable, but it immediately leaves you with no standard way to find the actual contents of a directory. From the point of view of a conforming Common Lisp program, the link files disappear, replaced by their targets. Standard programs are unable to discover link files, period. Most implementations provide some non-standard means of seeing the link files; I’m working on discovering these techniques. CL-FAD does not deal with this issue, which makes DELETE-DIRECTORY-AND-FILES fail sometimes, and makes it a pretty scary function in the general case, since it may follow symbolic links outside the tree that you’re trying to delete.

Links within the tree are interesting, too. You may encounter a link or its target in either order, with different consequences. If you delete the target first, the link becomes a broken link. If you encounter the link first, you better be sure that the program sees the link file itself and deletes it, rather than what deleting what it points to and leaving the link file behind.

In CLISP, there are two additional wrinkles to this problem. By default, DIRECTORY completely ignores broken symlinks. This behavior can be overridden with an implementation-specific keyword argument, and I sent Edi Weitz a change that adds this to CL-FAD’s LIST-DIRECTORY. (He immediately rolled this out in 0.6.1. Thanks, Edi.)

Worse, although CLISP offers a way to list symbolic links (the non-standard :FULL keyword of DIRECTORY), it only works with links to files and broken links, and not with links to directories. I may submit a bug report for this, because I haven’t found a way around it. Symbolic link files that reference existing directories appear to be invisible to a CLISP program.

The next (and perhaps last) problem, if you can solve the preceding ones, is deleting what you find in a directory. Again we have the (solvable) problem of variations in DELETE-FILE’s ability to delete directories. But there’s also a potential issue with deleting a symbolic link file. I think most Lisps will allow you to delete a symbolic link using DELETE-FILE. However, I have found that under certain circumstances, CLISP cannot. (And for this I did submit a bug report.)

I picked on CLISP a lot above, but it happens to be my first target for getting the unit tests running. I don’t yet know how I’ll fare with other Lisps — or how many of them I’ll have the patience to test.

Leave a Reply

© 2017 Mulling Things Over | Entries (RSS) and Comments (RSS)

Design by Web4 Sudoku - Powered By Wordpress