Enabling "foreign" callbacks into the LispWorks IDE


 

Abstract

This document explains how to "rebuild" LispWorks for Windows in such a way that threads which weren't created by Lisp can call back into Lisp.

The text only applies to Lispworks 4.4.6 and you'll need the Professional or Enterprise Edition if you want to imitate what's shown here.


 

Contents

  1. Why would you want to do this?
  2. Disclaimer
  3. The Lisp part
  4. The C part
  5. I'm too lazy
  6. Acknowledgements

 

Why would you want to do this?

LispWorks uses native threads on Windows but usually you can only call back into Lisp in a thread that was created by Lisp. According to the documentation, you can circumvent this if you deliver a DLL but it is not really clear (at least to me) if and how one can use the IDE for interactive development of programs (DLLs) that have to deal with such "foreign" callbacks.

Well, it turns out that you can use SAVE-IMAGE to save a Lisp image as a DLL and once you've figured out how to start this DLL from a small C "driver" you can have your cake and eat it too. This is what the rest of the document is about.

Note: LispWorks 5 doesn't need this workaround anymore.
 

Disclaimer

Of course, if you do what I describe here, you do it at your own risk. I take no responsibility for anything that you may do as a result of reading this page the contents of which are provided 'as is' with no warranty. Yada, yada, yada...
 

The Lisp part

First, you have to create a file with Lisp code that'll take care of creating the DLL image. Mine looks like this:
(in-package :cl-user)

(load-all-patches)

;; customize here
(let ((cwd (get-working-directory)))
  (load "/home/.lispworks-full")
  (change-directory cwd))

(push '("Environment startup" nil env:start-environment)
      mp:*initial-processes*)

(fli:define-foreign-callable ("StartLisp" :calling-convention :cdecl)
    ())

(save-image "lispworks-4460.dll" ; the name of the DLL
            :dll-exports '("StartLisp")
            :console :input)

(quit)
The second form (below the orange comment) is optional. This is what I use to load LW-ADD-ONS into the image and obviously you can load whatever you want at this point.

Now open a Windows command prompt and enter something like the following:

cd "c:\Program Files\LispWorks"
lispworks-4450.exe -init create-dll.lisp
(This obviously assumes that your LispWorks installation is in c:\Program Files\LispWorks and that you saved the Lisp code above under the name create-dll.lisp in that directory.) Now you should have a DLL named lispworks-4460.dll there.
 

The C part

For this section you'll need a way to compile a little C/C++ program on Windows. (But see the next section.) I used Microsoft Visual Studio .NET 2003 but I think the free Express Edition of Visual C++ should suffice.

First we need a DEF file for our DLL. That's simple - just create a text file lispworks-4460.def with these two lines in it:

EXPORTS
        StartLisp
and put it into your LispWorks directory. Now we create a LIB file from this DEF file: In the same command prompt window as above enter the following line
lib /def:lispworks-4460.def
You'll see a warning that you can ignore but as a result you'll get two new files, lispworks-4460.lib and lispworks-4460.exp.

Now start Visual Studio and create a new C++ Win32 console project. There, in the file stdafx.h add this line at the end:

#include <windows.h>
The code file that initially contains the stub for your _tmain function should consist of the following code:
#include "stdafx.h"

extern "C" {
  __declspec(dllimport) void StartLisp(void);
}

int _tmain(int argc, _TCHAR* argv[]) {
  StartLisp();
  FreeConsole();
  Sleep(INFINITE);
  return 0;
}
Then go to "Project" > "Properties" > "Linker" > "Input" and add "lispworks-4460.lib" under "Additional Dependencies." Finally, copy or move the file lispworks-4460.lib from the LispWorks folder to your project folder for this Visual Studio project.

You should now be able to build an EXE file. Put this EXE into your LispWorks folder (where you still have the DLL lying around that you created in the last section) and start it by double-clicking it. This should start the familiar LispWorks IDE but now foreign threads should be able to call into Lisp.
 

I'm too lazy

If you're too lazy for the C part you can download the EXE I created from http://weitz.de/files/LW.exe (about 23 kB) after you've read the disclaimer. It'll look for a DLL called lispworks-4460.dll that exports a C function StartLisp with no arguments.
 

Acknowledgements

Most of the above was explained to me by Nick Levine. I just fiddled a bit with Visual Studio and wrote it all down.

$Header: /usr/local/cvsrep/lw-callbacks/index.html,v 1.7 2006/08/28 10:16:33 edi Exp $

BACK TO MY HOMEPAGE