REPL Crash on Second Start

[hamzashahid-blit]

Hi, I used to use CEPL a lot. A month after I left it, it doesn't work the same.

## Let Me Explain
Right after creating my project normally with quickproject, cl-project or even CEPL's own cepl:make-project. I load the project and running cepl:repl works flawlessly, first try. On a restart of Sly, Slime or Emacs (all of which I have tested thoroughly on), on doing the same steps minus the project creation, then running cepl:repl yeilds a window which does not draw and does not respond (meaning it says "not responding"). It doesn't even give me back control of the REPL and instead presents me with a "REPL is busy" message. It also doesn't give me an error and the only option I have is to xkill the application.

## Steps to Reproduce
For this example, I am going to use the "Sly" REPL and the code is going to be from https://github.com/cbaggers/cepl.examples though this works on any code created with CEPL

1. git clone https://github.com/cbaggers/cepl.examples
2. cd cepl.examples
3. Open a new Sly REPL connection
4. (ql:quickload :cepl.examples)
5. (in-package :cepl.examples)
6. (load (compile-file "examples/game-of-life.lisp"))
7. (cepl:repl)
8. (run-loop)

Everything should work uptill now, display the window properly and work as expected. Time for the restart:

9. Close all instances of Emacs and/or REPLs
10. Run through steps 2 to 6 then follow the note below.

Between the bottom two, the order doesn't matter, because, as soon as Step 7 is run, the REPL averses from me.
7. (cepl:repl)
8. (run-loop)

This is my first propper issue, so please forgive my mistakes in formatting or otherwise. Thanks.

Edit: Add the restarting steps

[metayan]

Which OS and Lisp implementation are you using?

I just tested with SLIME/SBCL on macOS and here it works fine.
Replaced your line 6 with (load (compile-file "examples/game-of-life.lisp")).
Also, note that I have all the CEPL systems cloned in quicklisp/local-projects/.

The Lisp can be restarted with ,rest in the REPL – both in SLIME and SLY.

[hamzashahid-blit]

Oh I forgot to mention that, I am using Linux, and SBCL here is the details:

$ uname -a:
Linux Sabas-Zbook15 5.13.19_1 #1 SMP Sat Sep 18 18:18:26 UTC 2021 x86_64 GNU/Linux
$ sbcl --version
SBCL 2.1.9

I didn't know about the ,rest feature, that helps me alot. The replacement of line 6 doesn't make a difference on my machine. That was just a formal replacement for running C-c C-k on the buffer in Emacs. I also didn't know about the cloning on quicklisp.

Thanks for the reply.

[hamzashahid-blit]

Just making it clear that it still doesn't work as expected. :)

[metayan]

Thanks for clarifying that it's still not working. I wasn't sure. :)

1. It's still not completely clear to me what the actual issue is. What do you mean with "the REPL averses"?

2. Do you have something in ~/.swank.lisp or ~/.slynk.lisp?

3. I'm mostly using SLIME, so it might help if you can use that while trying to fix this.

Note:
It might be easier to help with the issue if you supply "Steps to reproduce" that you actually use.

The replacement of line 6 doesn't make a difference on my machine. That was just a formal replacement for running C-c C-k on the buffer in Emacs.

Everything makes a difference. We're dealing with computers here. ;)
Plus, line 6 could not possibly work.

CEPL is relying on quite many things in the system to work.
It's brilliant, and very much worth the effort to get it running well, but it might need some special attention.
I'm using it on both Linux and macOS, and it's much easier to get running on Linux, so I'm pretty sure we can get it running on your system as well. 🚀

[hamzashahid-blit]

Thanks for replying.

1. (a) The actual issue is elaborated on the Let me explain section but here goes:
   I create a project, it works as expected. I open up the project again the next day after a restart of everything. On running (cepl:repl) a window appears which is not responding. I don't have any control of the REPL anymore, meaning when I type anything into the repl, it just gives me a REPL is busy message. The only option is to run xkill and terminate the application.

2. (b) "the REPL averses" means that I loose all control of the REPL and have to terminate it.

3. I don't have a ~/swank.lisp nor a ~/slynk.lisp. I do have a ~/.sbclrc though: http://ix.io/3Cpw

4. Okay, I will use Slime from now on but I have tried it on SLIME too and it has the exact same results.

In your Note:

It might be easier to help with the issue if you supply "Steps to reproduce" that you actually use.

I actually used those steps. I copied from the REPL and double checked it by restarting and typing in those same steps. I tried both, C-c C-k AND Step 6 (Note the new one typed below). It doesn't make a difference. I am having the same problem.

Plus, line 6 could not possibly work.

Yes here is the new Step 6:
6. (load (compile-file "examples/game-of-life.lisp"))
I updated my original comment to the correct step 6.

I totally agree with the last paragraph and most of my projects depend severely on CEPL. I love it too much.

Thanks for cooperating. :)

[hamzashahid-blit]

Just another reminder that my problem isn't fixed. Am I not giving some information, please say so, I am sorry if that is the case. If my problem is fixed I will definitely state it. Thanks for your cooperation and I hope I am not a trouble.

[metayan]

No trouble. Pardon the delay. I'm still using macOS mainly, so couldn't test on Linux yet.

Do you have the CEPL repositories cloned locally, or are you using the ones that Quicklisp downloads automagically?
Asking because of the NOTE at the bottom of https://github.com/cbaggers/cepl.examples . Not sure how relevant it is anymore, but it could influence something.

Anyway, I would first test the system this way to make sure it works without SLIME:

cd ~/quicklisp/local-projects/
git clone https://github.com/cbaggers/cepl.examples
sbcl --eval '(ql:register-local-projects)' --quit

While in ~/quicklisp/local-projects/ , it might be good to

grep cepl.examples system-index.txt

and check that cepl.examples only appears once. (I've seen repositories that contain duplicate systems in subdirectories, and that can cause very strange errors.)

Now

cd ~/quicklisp/local-projects/cepl.examples/
sbcl --no-sysinit --no-userinit \
  --load ~/quicklisp/setup.lisp \
  --eval '(ql:quickload :cepl.examples)' \
  --load examples/game-of-life.lisp \
  --eval '(cepl:repl)' \
  --eval '(cepl.examples::run-loop)' \
  --quit

should run the example fine. Twice. ;)
(Starting SBCL this way to make sure that no init files introduce unexpected behavior.)

This is also a useful test:

sbcl --no-sysinit --no-userinit \
  --load ~/quicklisp/setup.lisp \
  --eval '(ql:quickload :cepl.examples)' \
  --load pre-release-run.lisp \
  --eval '(cepl:repl 800 600)' \
  --eval '(cepl.examples::run-em-all)'

To try to expose your issue, I would run this twice.

If that works – and can be run twice without problems – the next step would be to try with init-files:

sbcl \
  --eval '(ql:quickload :cepl.examples)' \
  --load examples/game-of-life.lisp \
  --eval '(cepl:repl)' \
  --eval '(cepl.examples::run-loop)'

and

sbcl \
  --eval '(ql:quickload :cepl.examples)' \
  --load pre-release-run.lisp \
  --eval '(cepl:repl 800 600)' \
  --eval '(cepl.examples::run-em-all)'

If all this works, then I'd start SLIME and hopefully it will work the same. Also twice.

[hamzashahid-blit]

Do you have the CEPL repositories cloned locally, or are you using the ones that Quicklisp downloads automagically?
Asking because of the NOTE at the bottom of https://github.com/cbaggers/cepl.examples . Not sure how relevant it is anymore, but it could influence something.

I used quicklisp and didn't clone it locally.

cd ~/quicklisp/local-projects/
git clone https://github.com/cbaggers/cepl.examples
sbcl --eval '(ql:register-local-projects)' --quit

This ran successfully.

Here is the grep output:

$ pwd
/home/saba/quicklisp/local-projects
$ grep cepl.examples system-index.txt
cepl.examples/cepl.examples.asd

Runnning:

cd ~/quicklisp/local-projects/cepl.examples/
sbcl --no-sysinit --no-userinit \
  --load ~/quicklisp/setup.lisp \
  --eval '(ql:quickload :cepl.examples)' \
  --load examples/game-of-life.lisp \
  --eval '(cepl:repl)' \
  --eval '(cepl.examples::run-loop)' \
  --quit

Gave me a CEPL:REPL window that showed up again not drawing anything and having the same symptoms as before. Although we have new information in the terminal, this shows up:

Running it a second time, yielded successfull results. A window running the game-of-life example as it should! This was the last time I saw anything properly rendered onto the CEPL:REPL screen. I ran all of the examples after that atleast 3-4 times. All of them produced the same output and the same window as shown in the picture above (They didn't work).

Atleast we now (hopefully) know a bit more about the issue.

Thanks.

Regards,
Hamza Shahid

[metayan]

(When there's text output, it's nice to paste only the text, because then one can copy from it.)

Looks like it's neither CEPL nor SBCL causing the problem.
Searching for xcb_xlib_threads_sequence_lost found these:
https://github.com/avafinger/libx11_1.6.4/blob/6f5c84b2cf20aa5a43d7c26078d8e6f7693f7af1/README.md
linuxmint/muffin#583
https://bugs.launchpad.net/ubuntu/+source/gtk+2.0/+bug/1808710
https://gitlab.freedesktop.org/xorg/lib/libx11/-/merge_requests/34

Apparently the problem appears quite randomly for others as well.

Seems to be time for a system upgrade. ;)
It might be useful to post the current contents of your /etc/os-release and the current version of libx11 here as well for future reference.

When running in SLIME, the error is probably also shown in the *inferior-lisp* buffer. Just mentioning it as another place to look for clues.

Time to enjoy some CEPL!

[hamzashahid-blit]

(When there's text output, it's nice to paste only the text, because then one can copy from it.)

Sorry, will keep that in mind for next time.

$ cat /etc/os-release 
NAME="void"
ID="void"
DISTRIB_ID="void"
PRETTY_NAME="void"

Here is my version of libX11 and it's deps (dependencies):

$ xbps-query libX11
architecture: x86_64
automatic-install: yes
filename-sha256: 817a48f06f4b524a808a0ed905d72d634e3a0f026815287abf34f646bf89b9d3
filename-size: 1318KB
homepage: http://xorg.freedesktop.org/
install-date: 2021-07-31 14:50 PKT
installed_size: 6909KB
license: MIT
maintainer: Leah Neukirchen <leah@vuxu.org>
metafile-sha256: d51441be21283779e09302200b568f1166b3fb8501b2c8544c386b9a56e8f06f
pkgname: libX11
pkgver: libX11-1.7.2_3
repository: https://alpha.de.repo.voidlinux.org/current
run_depends:
        libxcb>=1.2_1
        glibc>=2.32_1
shlib-provides:
        libX11.so.6
        libX11-xcb.so.1
shlib-requires:
        libxcb.so.1
        libdl.so.2
        libc.so.6
short_desc: Base X libraries from Xorg
source-revisions: libX11:6acfca310e
state: installed

I updated my system, restarted it and it still produces the same result. libX11 and deps are the same version meaning they aren't updated. Maybe I have to wait for it to be the latest version on Void Linux or I could build from source?

So still no time to enjoy CEPL... :(

[hamzashahid-blit]

Hello, Hope you are well,
I see that the version of libX11 I have is the latest version. Arch Linux, Void Linux, GitHub, all have libX11 version 1.7.2_3. I have searched more for this problem without any fix. Although, I went into the LDB Debugger and typed in backtrace which produces this output:

ldb> backtrace
Backtrace:
   0: fp=0x7fb0a388eea8 pc=0x7fb0a373cf81 Foreign function gsignal
ldb> 

Maybe this is of help to you?

Thanks for the help and I hope we could solve this together,as if we don't, I will have to use cl-opengl and create my projects from scratch with that, which is quite a bigger pain since I know that CEPL exists.

Regards,
Hamza Shahid :)

[metayan]

I have a suspicion that cl-opengl might behave the same way if libX11 is the culprit.
If the latest libX11 still has problems, I would downgrade it to the one that worked.
Or maybe it's mesa that is the problem and a fix seems to be on its way:
https://gitlab.freedesktop.org/mesa/mesa/-/issues/4763
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12006

Might be an idea to run the SDL2 tests from https://github.com/libsdl-org/SDL .

mkdir build; cd build; cmake .. -DSDL_TEST=1; make

to build the tests. On my system I had to add

C_INCLUDES = -I../include

in build/CMakeFiles/SDL2_test.dir/flags.make to get rid of

src/test/SDL_test_assert.c:28:10: fatal error: 'SDL_config.h' file not found

Most likely we can rule out both CEPL and SBCL as the cause.

[hamzashahid-blit]

Wow, all of the SDL Examples/tests work perfectly fine!

I tried the CEPL examples again and they produced the exact same results as before. Which means, the problem is in CEPL... Ooor cl-opengl?

To test cl-opengl, I first git cloned https://github.com/3b/cl-opengl, then went to the examples directory. After I ran

(ql:quickload :cl-opengl)
(ql:quickload :cl-glut)
(ql:quickload :cl-glu)

I did a C-c C-k on the examples/redbook/movelight.lisp file. On running the command (rb-movelight), this showed up:

Yay! It worked! I tried this on almost all examples, all of the ones I tried, worked flawlessly. What can we rule down?

1. cl-opengl
2. an OpenGL issue
3. my Hardware
4. my OS/Software
5. libX11 (atleast for the above examples we can rule this out)
6. SBCL
7. SLY
8. SLIME (I tested it on SLIME too, aside from a small package issue it worked the same as SLY)

Would you look at that, we narrowed CEPL down as the verdict (atleast as far as I can think).

This session boosted my motivation a little further as atleast I could see a window popup. Now the question would be where CEPL is going wrong.

Thanks for your cooperation.
Regards,
Hamza Shahid

[metayan]

I can try with a Funtoo Linux system later. I'm planning to switch to it from macOS at some point, so have to make sure that the graphics things – especially CEPL – work well. It has worked flawlessly on that system in the past.

It's good if you take note of any error messages that appear.

[metayan]

All the CEPL examples run fine on Funtoo Linux here.
cepl.examples in local-projects, dependencies directly from quicklisp.
libX11 1.6.9 - libX11.so.6.3.0 installed 2021-08-17
mesa 20.2.4