The Web Graph Database

wiki:Docs/ProbeFramework/DevelopmentProcess

Probe Framework Development Process

An InfoGrid Probe is a simple Java class with the responsibility to parse an input data stream (or, in case of ApiProbes, access a certain API) and to create suitably blessed and related MeshObjects that capture the information obtained from the data source.

Developing a Probe typically takes the following steps:

  1. Determine if there is already a Probe that serves the desired purpose, or that can be adapted to serve that purpose. InfoGrid already contains a number of Probes that can typically be found in sub-packages of the org.infogrid.probe package. These sub-packages exist in a number of different InfoGrid modules.
  2. Determine the semantic Model needed to accurately capture the meaning of the information that you are accessing. InfoGrid contains a number of models that can be used. They typically are found in projects starting with org.infogrid.model.
  3. If adequate EntityTypes, RelationshipTypes and PropertyTypes have not yet been defined for the requirements of representing the content of your data source, you need to decide whether and how to create or extend/modify one or more Models. You have three choices:
    • You create an entirely new Model.
    • If you already have use a non-InfoGrid Model, if that Model is defined by you, and only your own code depends on it, change it any way you want, at any time. Of course, you need to make sure that you take into account all ramifications of the changes, just like for any other software modifications that you might be making.
    • If it is someone else's Model, add your additional semantics to what is available already by creating your own new Model that relates to the existing EntityTypes and RelationshipTypes any way you want: e.g. through subtyping, through new RelationshipTypes, or by adding new EntityTypes that relate to the existing EntityTypes. Never change someone else's Model. You create a new Model by creating a new ModelModule.
  4. Make the modifications and run the InfoGrid code generator, typically by running the code generation part of the default InfoGrid build for the ModelModule.
  5. Determine whether or not your data is being accessed as a stream (like a web browser accesses an http URL, for example), or through an API (by querying a database, for example). Depending on that, and whether or not a data source is in XML format, write a new Java class that implements:
    • org.infogrid.probe.xml.XmlDOMProbe (your Probe works on a pre-parsed XML Document Object Model obtained through a stream);
    • org.infogrid.probe.NonXmlStreamProbe (works on an input stream directly), or
    • org.infogrid.probe.ApiProbe (accesses a data source through an API rather than a stream).
  6. Implement the Probe class. Regardless which interface you choose to implement, it requires the implementation of a constructor that does not take any parameters. For an XML DOM implementation, implement method: parseDocument. For non-XML stream data, implement method: readFromStream. When accessing information through an API, implement method: readFromApi.
  7. Add your new Probe to your application's main StandardModule, or create a new StandardModule that contains your new Probe class and all other files that it may need. This Module needs to specify dependencies on the InfoGrid kernel and all ModelModules that it uses. Add the new Module to the build. Generally it is not advised to put the Model and a Probe instantiating the Model into the same Module; if they are in different Modules, alternate Probe implementations can easily instantiate the same Model, which is helpful from a project organization perspective.
  8. Add your new Probe to the ProbeDirectory for your application, and specify which mime types, protocols or URLs it should handle.
  9. Run the build and test.
Last modified 5 years ago Last modified on 01/19/10 16:50:43