In Which I Play ChatGPT5 in Chess
Friendly reminder that I’ll be hosting a livestream today, Friday October 10, starting at 5pm EST with Jennifer Shahade. We’ll be discussing chess-playing LLMs. To join the stream live, where you’ll have the chance to participate and ask questions, download the Substack app and make sure to follow me so you’re notified when we go live.
When Jennifer said, “Every time I play ChatGPT in chess I learn something important about human thinking too,” I knew I had to try a game.
The first thing I noticed is that ChatGPT is quite sycophantic. It gushed praise on all my moves, starting from 1. e4. It did, however, offer some relevant and coherent commentary on the game, although it did get confused from time to time. Competitively speaking, it played quite well until blundering with 18…d5. Here’s the game in full:
What was more interesting than the game, however, was our chat. You can read my full, unredacted interaction with ChatGPT here:
Here’s a little extract:
Hope to see you at the livestream!
This is the way tto go. Crticial thinking and the chat bots not as ghost writers but as the object of the discussion. I have not read yet. Well, just surprised it is still sycophantic, I hear people who liked that before being displeased with that marketting parameter adjustment or module having been toned down. Maybe that is a way to throttle expectations while waiting for more ressource hungry behemoths server farms. They are kind of ponzee scheming all this investment construct, always ahead of anything tangible....
Anyway I welcome you published point of view on this. Good idea. see how it fares now.
So, using only PGN text matter, as its training sampling model (guessing as part of the big blob training corpus) it can now guess good moves. Did you check with existing databases about the parts it got right? Also I notice the messaging throught he conversation... So you are the injector now through that conversation. I am losing my spaghetti of a stream here.
So, was it saying mediocre, professional sounding annotations.. or were they chess appropriate and supported by arguments, not gratuistous opinions. I would not know, withoiut making a study of the game.. and then mapping the conversation and studying myself. Would welcome your first weeding trhough. or pointing at the accurate supported informative versus the possible still plausible but not that informative. ( I understand that might be in the eye of the beholder, so asking your eyes).
I am just noticing that you are able to embed a lichess object here. Is that within substrack or some web embedding through html? Anyway, it makes it a bit hidden to some of my web extensions, that I could use directly on lichess as the web page (if not Substack that might be why), such as save text as file. but the download button on the PGN workds. Thanks for the find.