NLP at work in AlchemyAPI.com

July 31, 2012 at 10:57 AMAdministrator

I ran across AlchemyApi about a year ago and implemented a service that digested all the feeds I'm subscribed to in Google Reader and saved meta data about each feed post in a database.  This allowed me to search the database from a Silverlight front end and look at posts with a given keyword, category, etc.

What is AlchemyAPI?

This is an API to which you can pass text or the URL to a web page and what you get back, depending on the service method you call, are entities, keywords, categories, concepts, relation and more.  The service uses NLP to process the text or web page and extract the asked for values. 

How is it useful?

It would seem to me the primary use for such functionality would be to have it parse the text of a piece of content and the returned meta-data would be used to tag certain words or phrases such that a user could hover over the text or click it and see information about it.  Text specifying companies, people or technologies, for example, could contain information and links to further define the item

Since the service returns keywords and categories, it's also useful for the classification of a body of text.  This is what I plan to use the service for.  I am subscribed to over 500 feeds in Google Reader and it takes a considerable amount of time to browse all the posts for a given day.  I'm currently working on a project that will process all the posts from each feed and save the meta-data about each to a database.  This should, effectively, create a knowledge store I can search from a web application in order to research topics of interest.  I will also be using it to expand the information my smart home knows about by mapping the entities to tokens inside the Language parser.  Also, the relations (RDF triplets) form statements that can be retrieved when the user asks a question of the smart home system.  I'm rewriting the implementation of this application to provide this integration and also to take advantage of TPL Dataflow to parallelize the processing.

Details

By just signing up, you get 1000 API calls a day.  They offer plans to support whatever level of service you require.  The services are implemented as web API's, but they provide a set of .Net classes that simplify making the calls.

In the next few posts, I'll be writing about details of getting set up to use the service and how I'm using it in the application I mention above.  I'll also be sharing the code for this application when it gets to a point where it makes sense to do so.

Posted in: .Net | Natural Language Processing | Smart Home

Tags: , ,

Smart House Alerts

August 25, 2011 at 9:15 AMmgordon

Bad weather on the way? My Smart House let's me know. I have an appointment this afternoon? It lets me know. A favorite blog has an update? Yep.

I decided early on that I wanted my smart house to notify me of some things. As a result, the time I used to spend checking the weather site, checking for blog updates, checking my calendar is now free time. These things are pushed to me, now, by my smart home.

The system has a series of alerts that do little more than poll various end-points or database records on timers. When conditions are such that I need to be notified, I get a message.

How does the application know how to reach me? Well, a couple of ways. In my system, there is a concept of a communication channel. This channel is associated with a particular user of the system and a certain mode of communication (IM, SMS, email, more to come). When I log into IM, for example, the system knows I'm online and that channel is marked as being open. If an alert occurs, it knows I can be reached on that channel.

For other channels, such as email and SMS, the channel always remains open. For these, I can configure the application to notify me on that channel or not depending on the alert.

IM

I use the XMPP protocol for IM communication with my Smart House. This allows me to use Google Talk or Jabber to get messages from or send messages to the system. I have taken a dependency on the the XMPP SDK from Ags. It allows me to connect to the network, to be notified when any of the Smart House's buddies go on or off line, and to send and receive messages.

Connecting is simple. First, instantiate and configure a connection.

Connect to XMPP
  1. Conn = new XmppClientConnection(ConfigurationManager.AppSettings["XmppServer"]);
  2. Contacts = null;
  3. Contacts = new List<Contact>();
  4. Conn.EnableCapabilities = true;
  5. Conn.ClientSocket.ConnectTimeout = 60000;
  6. Conn.AutoResolveConnectServer = false;
  7. Conn.ConnectServer = "talk.google.com";
  8. Conn.Port = int.Parse(ConfigurationManager.AppSettings["XmppPort"]);
  9. Conn.UseStartTLS = false;
  10. Conn.UseSSL = true;

Then, add some event handlers.

Set Event Handlers
  1. Conn.OnError += conn_OnError;
  2. Conn.OnMessage += conn_OnMessage;
  3. Conn.OnPresence += _conn_OnPresence;
  4. Conn.OnLogin += s =>
  5. {
  6. IsConnected = true;
  7. var p = new Presence(ShowType.chat, "Online") { Type = PresenceType.available };
  8. Conn.Send(p);
  9. };
  10. Conn.OnAuthError += (s, e) => StructuredSpeech2.Logging.Logging.Log("XMPP Authentication Error");
  11. Conn.OnRegisterError += (s, e) => StructuredSpeech2.Logging.Logging.Log("XMPP Register Error: " + e.InnerXml);
  12. Conn.OnSocketError +=
  13. (s, e) =>
  14. StructuredSpeech2.Logging.Logging.Log("XMPP Socket Error: " + e.Message + " :: " + e.StackTrace);
  15. Conn.OnStreamError += (s, e) => StructuredSpeech2.Logging.Logging.Log("XMPP Stream Error: " + e.InnerXml);

Finally, connect.

Connect
  1. try
  2. {
  3. Conn.Open(ConfigurationManager.AppSettings["XmppUsername"],
  4. ConfigurationManager.AppSettings["XmppPassword"]);
  5. }
  6. catch
  7. {
  8. Initialize();
  9. }

Notice that many of the event handlers we're handling have to do with various things that can go wrong. There are logged for investigation later. The OnLogin event fires when a connection to the network has been established. Here, I'm setting the system's status to online and available.

When a login fails, we recurse and try again.

The OnPresence event fires when the presence status of a user has changed such as their becoming available or signing off. The event args let us know the buddy and what their new presence value is. When I handle this event, I evaluate the new presence value and then do two things. I keep a collection of all online contacts so I can quickly broadcast a message. The first thing I do when I get a presence event is to update this collection by adding or removing the contact as appropriate. The second thing I do is update the state of the communication channel for the user the buddy represents.

The OnMessage event fires when a message has been received from a buddy. The event arguments specify the the buddy the message is from and the text of the message. When a message is received, I check to see that the buddy is a user on the system and if they are, I process the message with the Language Processor. If the user is not found, a response is sent to the buddy stating they are not a known user and cannot interact with the system.

Finally, I can send a message to any online buddy by calling the send method on the connection class. As with the the previous discussions about the Speech Processor, the entire conversation is logged and used in processing the text sent in a particular message. In the case of IM, the conversation starts when the buddy connects to the network and ends when they disconnect. When a message has been received, the Language Processor processes it and then uses this send method to reply to the appropriate user. Alerts are send in the same way.

The bottom line is that I can send commands to my house via IM and have it act appropriately or it can send me messages to alert me if my attention is needed. A true conversational interface.

Email

I'm using OpenPop to provide the functionality need to check and manipulate emails on the POP server. My Smart House has it's own email account which it polls on a timer. Each time, all messages are pulled from the email account, processed and then deleted. OpenPop allows the system to connect to the pop account and pull down all the messages. First, the connection object is created and configured.

Connect to POP3 account
  1. _client = new Pop3Client();
  2. _client.Connect(_popEmailServer, _popEmailPort, _popEmailUseSsl);
  3. _client.Authenticate(_popEmailUname, _popEmailPassword);
  4. _client.Reset();

Next, the messages are pulled down and processed.

Process Email Messages
  1. _emailCheckTimer.Stop();
  2. try
  3. {
  4. ConnectClient();
  5. if (!_client.Connected)
  6. return;
  7. int messageCount = _client.GetMessageCount();
  8. var messages = new List<Message>(messageCount);
  9. for (int i = 1; i <= messageCount; i++)
  10. {
  11. messages.Add(_client.GetMessage(i));
  12. _client.DeleteMessage(i);
  13. }
  14. List<Message> staticMessages = messages;
  15. var enc = new ASCIIEncoding();
  16. foreach (Message msg in staticMessages)
  17. {
  18. string request = enc.GetString(msg.FindFirstPlainTextVersion().Body);
  19. string fromAddress = msg.Headers.From.Address;
  20. User user = UserRepository.GetUserByEmail(fromAddress);
  21. if (user == null)
  22. {
  23. SendResponse("I don't know this email address.", msg.Headers.From.Address, msg.Headers.Subject);
  24. return;
  25. }
  26. if (_client != null)
  27. {
  28. _client.Disconnect();
  29. _client.Dispose();
  30. }
  31. CommandProcessor.ProcessCommand(request, user, ConversationMode.Email, fromAddress);
  32. }
  33. }
  34. catch (Exception ex)
  35. {
  36. StructuredSpeech2.Logging.Logging.Log("Checking mail " + ex.Message + ex.StackTrace);
  37. }
  38. finally
  39. {
  40. if (_client.Connected)
  41. _client.Disconnect(); ;
  42. _client.Dispose();
  43. }
  44. _emailCheckTimer.Start();

For sending emails, I use the SmtpClient class in the .Net Framework. See its documentation for usage.

SMS

Once email has been implemented, SMS is easy. I use SMS for outgoing messages, only. Most cell phone carriers allow SMS messages to be send via email. For example, an email sent to an address such as 10digitnumber@txt.att.net, will be received as an SMS message on the AT&T network phone having that 10 digit number. For this reason, I use exactly the same approach to send SMS messages as I do for sending emails.

Posted in: .Net | Smart Home | Natural Language Processing

Tags: , ,

Letting My Smart Home Keep a TODO List for Me

August 25, 2011 at 5:26 AMmgordon

In my last post, I summarized how I created a Natural Language Processor around which I am building a Home Automation solution. I want my smart home to be able to more than just turn things on an off, so I have been adding pieces of functionality that will automate more than that. The first piece of functionality I added was the ability to create and manage items on a TODO list.

Add the Tokens

If you'll recall from my last post, the language processor contains several Token classes that know how to parse out and tokenize words or phrases. These classes are the building blocks which are used to define the system's vocabulary. For the TODO list functionality, we need to define Token classes for the following:

  • "todo"
  • "list", "get", "show"
  • "create"
  • "delete"
  • "mark"
  • "done"
  • "for", "on"
  • dates and times

The language processor will parse the command we specify into a list of tokens that will be compared to the parameter lists of our command methods. When a match is found, that method is called. We need to define our command methods, next.

Create the Command Methods

In the end, we are wrapping little more than CRUD operations for a TODO list table in the database. One example of a command method is below.

Code Snippet
  1. public static void CreateNewToDoItem(ConversationContext cContext, TokenCreate create, TokenToDo todo, TokenQuotedPhrase phrase)
  2. {
  3. var repos = new ToDoRepository();
  4. var newTodo = repos.CreateNewTodo((string)phrase.Value, null, cContext.ConversationUser.UserId);
  5. cContext.Say("Added ToDo \"" + phrase.Value + "\"", newTodo);
  6. }

 

This method takes a token for "create", one for "todo" and another for a quoted phrase which is any phrase found between quotes. So an example of a command that might result in this method's being called would be

create a new todo "Call the hardware store about paint"

You can see that when the method is executed, a new record is created for the todo item. The first parameter is the conversation context. This context contains information such as what channel the communication is on (IM, email, etc.), the user being conversed with and a list of incoming and outgoing messages in this conversation. This is important in some cases. Consider when the user wants to delete a todo item. We could allow them to specify the item by its text, but that could be error prone and a lot of typing. It'd be better if they could specify only a number. So, in my implementation, the user requests a list of their items and what is returned is a numbered list. The user can then say, "delete 2" to remove the second item.

How does the system know which item was number 2? When the list is generated, it's stored in the context such that 2 corresponds to a particular item. From that point on, it doesn't matter what else happens to the records in the database because 2 doesn't just mean the second item, but the item that was second in that list at that point in time. This is possible because of the context.

The reply to the user gets sent to the right user through the right channel because of the context. In this implementation, we don't have to worry about these details in the command method, the context keeps it sorted out for us.

By repeating the process of creating the necessary tokens to build up our vocabulary and then creating rules to match, functionality is layered onto the system to allow the user to associate a due date and time with the item,

create new todo "call the insurance man" for tomorrow 1:30

Posted in: .Net | Smart Home | Natural Language Processing

Tags: , ,