If my life depended on timely blogging, I’d be dead ten times over. Regardless, there’s stuff to talk about, and it involves parsing text into graphs. I updated the dictionary we talked about in the last blog, and I’m currently in the middle of another update. The short story is that using lookups in the dictionary for parsing got me a much faster parse time, and I found yet more flaws in how I was representing the words, leading to another format and moving the dictionary into a database instead of a flat file that is parsed at startup.
Like a drunk who wakes up in the alleyway after a bender, I’m sitting here typing about some recent months of progress in natural language processing, especially where it concerns using a dictionary to help parse the language. It’s all still related to the Interrogative AI I’ve been working on, of course, and for conversational AI in general.
Basically, the problem I was facing was how to interpret what a player says to a character. At first, I resorted to simple actions that the NPC would filter through their personality traits to respond to, and that was okay. Then, I went a bit further, since I was implementing a text interface for a demo, and included a list of words and simple sentiment markup (-1 for negative, 0 for neutral, 1 for positive) that I’d find in statements.
That was less okay, as I quickly found that unrestricted text input makes for some statements that are open to interpretation. Then, I decided to look back to how I’m using attributes in the AI for inspiration. Maybe expanding the attributes to be a lot more flexible was an idea that will pay off better than I originally thought. Read on, my friends…
Time flies when you’re working on stuff…
The good news is that I have a demo coming for Interrogative AI, so that you can play with it. The bad news is that it’s not ready today. The good news is that the bad news is because it’s a multiplayer game with RPG capabilities. Because a static demo was just way too easy for me! The bad news? It’s still not ready today 🙁
Have a screenshot:
The current iteration turned out to be a good starting point for a much more flexible and resilient system. It’s slower than I’d like, with less flexibility and too many functions needed to make it work. It leans on too many things, and is not properly encapsulated. What Interrogative needs, in order to match the flexibility and power of its back-end querying and knowledge representation, is a Dialog Template system that is equally flexible, resilient to missing data, able to be localized easier, and much more simplified.
Okay, most of its glory. I finally settled on a color scheme for the editor, and after moving a few things around (and a few things yet to be moved), the editor is in a state good enough to show off. Take a look!
I have to admit that I had resisted the urge to change the tables in the database for a few months, mainly because I’m as lazy as anyone else, and all of my work so far had been done against the data structured as it was. However, I find it hard to think that people would like to buy an AI tool that asked them to add 20+ tables to their database, many of which had only a dozen or so lines of data in them. It was wasteful. And besides, there’s a few gains that get made with using 3-4 tables instead…
We’ve all told lies. From when you were a little kid trying to avoid punishment to the other day…when you were trying to avoid punishment. Lying is mostly about avoiding something that is harder. Avoiding judgement, punishment, awkward conversations, or more serious confrontations. Lies have a good deal of utility. Manipulating the knowledge you have can gain you an advantage over others, or mitigate certain circumstances. It is so prevalent in human society that most religious, cultural, and political power structures have varying amounts of laws, rules, and norms dedicated to when and why it is appropriate or inappropriate to lie.
Obviously, if we’re working on a conversational AI, we’re going to want to look at this behavior!
It’s been a while since my last update, mainly since I’ve been chucking so much time at two contracts, this project, and a newborn daughter, that every time I think of writing a blog post, my eyes turn to marbles. There’s a limit to what my brain’s been able to deal with over the past few weeks. However, with more sleep becoming available, and more progress being made on the (Windows) editor for Interrogative, I thought it was finally time for a blog post.
As a matter of fact, I have some screenshots to show off. A few shots of the Interrogative Editor, that allows you to access and edit the data that is used by the Personality-based AI, as well as the semantic knowledge that makes up the NPC knowledge for conversations. And live-game data, if you want- it works better when the NPCs have an inkling about the live game world they’re in. Interrogative doesn’t wall that off.
The last two blogs have focused quite heavily on my implementation of Dialog Templates, and for good reason: We’re working with live data and need fast, relatively easy ways to put that data into a format that fits conversations with players. Generally speaking, a lot of what we say can be broken down into templates. The majority of conversation tends to consist of agreements: “Okay.”, “Yeah”, “uh huh”, head-nodding, etc. The rest breaks down into a finite number of categories, which then break down into a finite number of ways you can get that information across to the other person. Granted, that finite number is incredibly large, but sentences aren’t just a permutation of words. Grammar, and the need to address certain topics, limits what you say in any given sentence dramatically, and being creatures of habit, people tend to talk in a certain “way”, and that “way” is just the format of words that they use. In short, Dialog Templates are built partially on how people use words.