Twenty years ago, my development team built a natural language processing engine that scanned employment, auto, and real estate advertisements for searchable categories. I knew that we had a difficult data management challenge. The data in some ad types were relatively straightforward, like identifying car makes and models, but others required more inference, such as identifying a job category based on a list of skills.
We developed a metadata model that captured all the searchable terms, but the natural language processing engine required the model to expose significant metadata relationships.
