Computer Pop Song Lyric Generation

Ben Bay — benjamin.bay@gmail.com — bayb2

Dan Ventura — Computer Science Department

I propose to write computer software that composes high-quality pop song lyrics. I will know my software is successful when unbiased observers deem its pieces to be creative and are unable to distinguish them from human-written pop pieces.

• Interest and demand for new music is limitless. It does not matter that there is already enough recorded music to last a human lifetime; people will never stop seeking new music. Lyric-generating software will help fill that demand.

• The profound complexities and mathematical patterns found in musical verse can be penetrated deeper than ever by computers. Innovative composition styles may come about digitally.

• For all history, songwriting has been viewed as a “humans-only” realm of artistry. When computers break this broadly-accepted notion it will shift the public eye, popularize computational creativity, and facilitate funding for the advancement of artificial intelligence.

• Pop is by definition popular, the music of the majority. It embodies the zeitgeist and reflects the movements of a time period. Pop is ubiquitously enjoyed and related to, and yet research for popular song lyric generation at the scale I am proposing is unprecedented (see Project Profile Body).

• Lyric generation is integral to the problem of generating complete pop songs. I will design my tool with this in mind; it will have the capacity to work alongside a pop music generator.

I will work closely with Paul Bodily, a PhD candidate whose emphasis is computational creativity in music [1]. I propose to integrate my lyric generation tool to his musical composition generator known as Pop* (Pop Star). I will distinguish my NLP (natural language processing) research efforts from past attempts through the use of careful big data practice, cutting-edge third-party software, and scholarly rigor.

• Data—For maximum lyrical potential, I will train my word-replacement models on large, high-quality corpora only. I will use texts from relevant writing categories such as lyric, spoken, fictional, magazine, and newspaper [2], and minimize encyclopedic or otherwise technical language input. Input text will be English, with corrected spellings. This data will be on a large scale (in the millions of texts and billions of words) in order to best model the immense complexity of language. I will use BYU’s supercomputing resources as necessary for data processing. I will primarily use this data for vector arithmetic: I will create a multidimensional vector space where each point represents a word. Clustering these points and finding the cosine distance between them will enable my software to understand and predict word relationships [3]. I will also extract n-grams, collocates, and word frequencies from the data. These will all be intelligently applied towards lyric generation.

• Lyrics by template structure—My initial approach to lyric creation will be descriptive rather than prescriptive; it will use already-existing pop songs as a guide. The software will pull a lyrical template from a database of over 1,000,000 pop songs, analyze its linguistic [4] and poetic structure, and use those structures to produce an entirely new set of lyrics.

• Lyrics by algorithm—After building a sufficient lyric replacement tool, I will build a tool that generates brand new song structures and lyrics algorithmically. To do this I will write a model for English grammar, modified for language in pop songs (i.e., to allow for certain types of grammatical errors).

• Rhyme—I will design and integrate a comprehensive rhyme system into my lyric generator. It will also work as a standalone tool. It will identify rhyme schemes, identify rhymes by their literary classifications, identify rhymes by their phoneme sequences (much like comparing nucleotide chains in genetics), and suggest new rhymes. It will draw from data on phoneme similarities [5] and employ rules established by experts in rhyme [6]. It will allow for complete user-customization; users will have complete control over any desired constraints or parameters for each of the tool’s various functions.

• I will integrate my lyric generation tools with Paul Bodily’s music generation tool, Pop*.

• I will publish my lyric generation tools and comprehensive rhyme tool on the web for anyone to try and use in their own projects.

• I will design and conduct a double-blind study measuring observers’ ability to distinguish computer-generated lyrics from human-written lyrics.

• I will write a scholarly paper on my final products including results from the above study, and submit it to the 8th International Conference on Computational Creativity.

My name is Ben Bay. I am a Computer Science major with an interest in imitative and experimental procedural content generation (creating data algorithmically), computational creativity, artificial intelligence, and deep learning. I have formal education in in big data, bioinformatics, algorithm design and analysis, software design and testing, computational and probabilistic models, discrete structures, data structures, Java, Python, C++, C#, and R. My current college GPA is 3.77. I am currently working at the Provo MTC as a software developer. There I build, debug, and maintain APIs for the TALL application, which focuses on language-learning. I also have an independent interest in literature, linguistics, and grammar. I have taken classes in comparative literature, essay writing, and political writing.

Dan Ventura is a Professor in the Computer Science Department at Brigham Young University. Prior to joining the faculty at BYU, he was a member of the Information Sciences and Technology division of the Applied Research Laboratory and a member of the Graduate Faculty of Computer Science and Engineering at Penn State University. Dan has also spent time in industry as a Research Scientist with fonix corporation, working on the development of state-of-the-art technology for large vocabulary continuous speech recognition. His research focuses on creating artificial intelligent systems that incorporate robustness, adaptation and creativity in their approaches to problem solving and incorporates neural models, machine learning techniques, and evolutionary computation.

December 2016—Lyric generator writes quality pieces

January 2017—Lyric generator integrated with music generator

February 2017—Paper submitted to ICCC-8

April 2017— Conducted study on discrimination between computers and humans as lyrical authors

June 2017—Software tools published online

1. Bodily, P. (2016). Computational Creativity in Popular Music Composition. BYU PhD Dissertation Proposal.

2. Davies, M. (2009). The 385 million word Corpus of Contemporary American English (1990–2008): Design, architecture, and linguistic insights. International Journal of Corpus Linguistics, 14(2).

3. Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. Proceedings of Workshop at ICLR.

4. Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S.J., & McClosky, D. (2014). The Stanford CoreNLP Natural Language Processing Toolkit. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 55-60.

5. Hirjee, H. (2010). Using Automated Rhyme Detection to Characterize Rhyming Style in Rap Music. Empirical Musicology Review, 5(4).

6. Pattison, P. (1991). Songwriting: Essential guide to rhyming: A step-by-step guide to better rhyming and lyrics. Boston, MA: Berklee Press.