AI is big news. Not a week goes by without a new development in the field. GPT-2, the second-generation language prediction created by OpenAI, is one of the AI projects getting a lot of that press. GPT-2 is seen as a major breakthrough in producing high-quality written content - so high quality that OpenAI initially feared the project was too dangerous to release. Luckily, OpenAI had a change of heart and released a modified version of the GPT-2 algorithm for the rest of us to get familiar with.
I had been wanting to work with GPT-2 for quite some time, and this was the perfect opportunity. What could be better than a content management system that writes the content for you?
GPT-2 is built with a default AI generator model, but it also allows for the creation of custom generators. I wanted to give users the option of choosing between several generated writing styles, so I built two additional custom generators to compliment the standard model. Building a really good generator requires a huge library of written material for the AI to learn from. Collecting and preparing these texts - the “corpus” - took several days.
Once the generators were built and running, I needed a way for Rails to communicate with them. As with everything else, there was a Gem available that made it easy. HTTParty made it simple to implement the interface, and the JSON structuring was a snap to parse and integrate.
Once the GPT-2 generators were prepared, I turned to building the Rails app. I used a straightforward approach with three primary models: Users, Documents, and Generators.
Each user can create many documents. Each document contains a writing prompt, parameters for the generators, and the response generated by the GPT-2 AI. For a prompt like this:
“In my younger and more vulnerable years my father gave me some advice that I’ve been turning over in my mind ever since.”
The GPT-2 generator would then continue the thought:
“If you don’t fall in love with your first love it will be too late. A good first kiss is the key to all of your future lovers, make sure to develop a good first kiss down pat because you will thank me in the end for the many times that I have spared your sorry ass from that huge headache you call a relationship.”
Fresh, right?
This structure provides a solid foundation to built the app on. The build process was a great chance to integrate nested resources, Active Record validations, error reporting, and scope methods. I enjoyed working with the Bulma CSS Framework for Foodbanks.nyc, so I integrated it into AICMS to give it a nice final touch.