The Web Graph Database

wiki:Docs/ProbeFramework/Example

Probe Framework: Implementing an example Probe

This example applies the Probe Development Process for an example Probe that knows how to read VCard files ("electronic business cards", commonly sent as e-mail attachments).

1. Determine if there is already a Probe that serves the desired purpose.

For this example, let’s assume there is not, although InfoGrid provides a VCardProbe. It is recommended that you look at the source code of this Probe as you go through this example.

2. Determine the semantic Model that is needed.

For this example, we need to be able to represent an address, a phone number and whether or not this phone number is a home, office or other kind of number. A real-world implementation would obviously support more concepts.

3. Determine how to instantiate / use / extend any existing relevant Model.

InfoGrid contains a VCard Model. (It is recommended that you look at its model.xml file as you go through this example.) This VCard Model can serve as an example for the Model that would have to be created if it did not.

As all Models, this Model is defined in a ModelModule, represented as a NetBeans project, here called org.infogrid.model.VCard. The Model itself is defined in file infogrid-models/model.xml relative to the NetBeans project's root folder.

When creating a new Model (just like when writing code or picking project names), make sure you create it with an identifier that uses your own organization's domain name. For example, if you work for the company that owns the domain example.com, you could create a Model called com.example.model.VCard. Do not use the infogrid.org domain name, unless you intend to contribute your code to the InfoGrid project.

This model.xml file defines a new SubjectArea that captures the Model. The SubjectArea, in turn, defines the needed EntityTypes, RelationshipTypes, and PropertyTypes. To highlight some:

  • an EntityType called VCard, which represents the "electronic business card" itself.
  • an abstract EntityType called Address, with two concrete subtypes: PhysicalAddress and CommunicationAddress.
  • a RelationshipType called VCard_Shows_Address that connects an Address to the VCard on which it is shown
  • suitable PropertyTypes, e.g. a PropertyType called FullName for EntityType VCard.

4. Make the modifications and run the code generator.

Having created the new NetBeans project and its model.xml file, some configuration files need to be adjusted to make the InfoGrid build process work, according to the instructions in Creating a new ModelModule.

Build the new NetBeans project.

5. Determine whether your data source is accessed through a stream or API and implement the corresponding interface.

The data source in this example is a non-XML VCard, so the to-be-implemented Probe class needs to inherit from interface NonXmlStreamProbe.

The InfoGrid-provided VCardProbe is called org.infogrid.probe.vcard.VCardProbe; if you had implemented it at example.com, you may have wanted to call it com.example.infogrid.project_foo.MySpecialProbe.

6. Implement the Probe class.

The new Probe class requires an empty, public constructor.

It is also helpful to import the relevant code generated by the InfoGrid Code Generator, using a statement such as:

import org.infogrid.model.VCard.PhysicalAddress;
import org.infogrid.model.VCard.VCardSubjectArea;

As this Probe accesses a stream that is not XML, method readFromStream needs to be implemented.

Defined in interface NonXmlStreamProbe, this method has the following signature:

public void readFromStream(
        NetMeshBaseIdentifier  dataSourceIdentifier,
        CoherenceSpecification coherenceSpecification,
        InputStream            stream,
        String                 contentType,
        StagingMeshBase        freshMeshBase )
    throws
        EntityBlessedAlreadyException,
        EntityNotBlessedException,
        IllegalPropertyTypeException,
        IllegalPropertyValueException,
        IOException,
        IsAbstractException,
        MeshObjectIdentifierNotUniqueException,
        ModuleException,
        NotPermittedException,
        NotRelatedException,
        ProbeException,
        RelatedAlreadyException,
        RoleTypeBlessedAlreadyException,
        TransactionException,
        URISyntaxException,
        StringRepresentationParseException;

Note that if the data source was XML, or an API, the corresponding method has a very similar signature and the following discussion applies correspondingly.

It helps to create the following helper variables in the method:

    MeshBaseLifecycleManager    life   = freshMeshBase.getMeshBaseLifecycleManager();
    MeshObjectIdentifierFactory idFact = freshMeshBase.getMeshObjectIdentifierFactory();

For this example, there is no point in discussing how to write a VCard parser; that is standard programming fare assumed to be known. Instead we focus on how to create the correct MeshObjectGraph for the information that was obtained from the VCard.

For each Address found in the VCard data source, a MeshObject needs to be instantiated that is blessed with the correct subtype of EntityType Address. The corresponding code may look like this:

    MeshObject newAddress = life.createMeshObject(
            idFact.fromExternalForm( "#abc" ));
    newAddress.bless( VCardSubjectArea.PHYSICALADDRESS );

    newAddress.setPropertyValue(
            VCardSubjectArea.PHYSICALADDRESS_COUNTRY,
            StringValue.create( "Mongolia" ));

    newAddress.setPropertyValue(
            VCardSubjectArea.ADDRESS_ISHOME,
            BooleanValue.TRUE );

Alternatively, you could use the generated Java interfaces for code that can be checked better against type errors at compile time, such as:

    PhysicalAddress newAddress = (PhysicalAddress) life.createMeshObject(
            idFact.fromExternalForm( "#abc" ),
            VCardSubjectArea.PHYSICALADDRESS )).getTypedFacadeFor( VCardSubjectArea.PHYSICALADDRESS );

    newAddress.setCountry( StringValue.create( "Mongolia" ));
    newAddress.setIsHome( BooleanValue.TRUE );

These two fragments are functionally equivalent.

In this example, a hard-coded local identifier is used as MeshObjectIdentifier for the newly created MeshObject of type PhysicalAddress. This of course only works if there is only one PhysicalAddress in any given VCard, which is unlikely. Refer to the example implementation for a better mechanism for how to create local identifiers.

Invoking the factory method, and then setting the values for the Properties like this is done for all MeshObjects to be created by a Probe except for one, and that is the ShadowMeshBase's HomeObject. Like for all MeshBases, the HomeObject is created by the automatically before the Probe is even run, and - like for any HomeObject – cannot be deleted. However, the HomeObject can (and usually, should) be blessed with the right EntityTypes and its Properties should be set to the right values.

The HomeObject is obtained from the StagingMeshBase passed into the method. It is then blessed appropriately according to the Model:

    MeshObject home = freshMeshBase.getHomeObject();

    home.bless( VCardSubjectArea.VCARD );

Each instance of Address that is created needs to be related to the HomeObject representing this particular VCard. Why? Because if we wrote a Probe that can read multiple VCards from the same data source, for example, not having the relationship would not make it obvious which Address went with which VCard. The Model defines a suitable RelationshipType with which the Relationship will then be blessed as follows:

    home.relateAndBless( VCardSubjectArea.VCARD_SHOWS_ADDRESS.getSource(), newAddress );

In these example invocations, we use hard-coded local identifiers, which only work if there is only one Address in this file, which is unlikely for VCards. Refer to the example implementation for a better mechanism for how to create local identifiers.

7. Add the new Probe to the right Module

This new Probe must be defined in a StandardModule that declares the ModelModule (containing the VCard SubjectArea) as a dependency, so the Module Framework can correctly resolve dependencies. This may be a StandardModule just for this purpose (like the InfoGrid-provided project org.infogrid.probe.vcard) or be part of the StandardModule that defines your application, or any other StandardModule that correctly declares its dependency on the corresponding Model.

8. Add your new Probe to the application's ProbeDirectory.

The ProbeDirectory is specified when creating a LocalNetMeshBase or ShadowMeshBaseFactory. The relevant code may look as follows (compare with the VCard Probe test project org.infogrid.probe.vcard.test):

    MProbeDirectory theProbeDirectory = MProbeDirectory.create();

    theProbeDirectory.addStreamProbe( new StreamProbeDescriptor( "text/x-vcard", VCardProbe.class ));

    ShadowMeshBaseFactory theShadowFactory = MShadowMeshBaseFactory.create(
            ...
            theProbeDirectory,
            ... );

9. Now build, and test.

Hopefully, everything works. When a URL is accessed that points to a data source of the VCard type (check your web server’s or operating system’s mime types and/or extensions), the Probe Framework should access the data source, invoke the newly written Probe class, create a single instance of VCard and zero or more instances of PhysicalAddress and CommunicationAddress.

Further, it should continue polling that data source and if you modify the data source (e.g. by copying a different VCard to the same URL, or by editing the VCard file), the Probe Framework should automatically update the VCard and/or Addresses in the main NetMeshBase.

To try out this behavior, either:

  • run the automated VCard test provided as part of InfoGrid. It is defined in project org.infogrid.probe.vcard.TEST. To understand how it works, it may be advantageous to run this project in the debugger and step through the Probe.
  • run NetMeshWorld. Note: NetMeshWorld does not currently include the VCardProbe, but you can try a standard HTML URL instead.
Last modified 4 years ago Last modified on 01/19/10 16:53:12