Text Analysis and Natural Language Processing on the Semantic Web

Although Natural Language Processing (NLP) has been around since the 1950s in the computer science world, more and more uses for this powerful technology are being uncovered every day. Search engines like Google use NLP as one of the ways they extract meaning from web pages, Microsoft has a whole team of people working on NLP projects, and a number of universities have dedicated major resources working on the advancement of NLP, but what about everyone else?

NLP has many uses going beyond behemoth websites including uses for the enterprise, small business, and end users. In this article, we’ll be taking a look at some of these use cases and key players in the field.

What is Natural Language Processing?

At its core, Natural Language Processing involves a machine trying to extract meaning from real language by identifying mood and meaning, relationships, common patterns, and recognized terminology. Although the design of the algorithms is very advanced, the use cases don’t have to be.

Some practical applications are very accessible to most web developers, and several of the vendors listed below have showcases of official or user-submitted projects that use their service. Projects include:

Content Tagging and Suggestion
Social Media Analysis
Data Visualization
Geotagging
Mood Analysis
SEO
Content Discovery and Sharing

Get Started by Choosing a Service

Once you decide to use language processing, you need to work out the project requirements and the details of the implementation. Some services are better suited to certain tasks than others and all of the following are factors in the selection process:

Do you want a web service or command line tool?
What are you processing: text, documents, web pages?
What information do you want back from the analysis (i.e. keyword extraction, sentiment/mood analysis, etc)?

Web Services

For purposes of this article, I am outlining vendors that allow commercial use, offer a free subscription level, and also have premium subscriptions should your project move beyond a proof of concept.

Alchemy API

Diffbot

Extractiv

Open Calais

Repustate

Saplo

Zemanta

Command Line

Natural Language Tool Kit (Python)

Topia (Python)

Conclusion

Natural Language Processing is an up-and-coming technology that people are discovering new uses for every day, and if you understand a scripting language, you should have no problem working with Natural Language Processing technology. Most of the vendors even offer Software Development Kits (SDK) in languages such as PHP, JavaScript, Java, and Python to help you get started using their product in your web development projects.