Before Thanksgiving, we created the model that lets us represent people in MyHealth. Now let’s add some people, particularly my family, because that’s what I’m building MyHealth for in the first place.

First, we’ll replace the front Viewlet with one that shows a list of people whose data is being managed. So we open up MainViewletFactory.java and make its determineFactoryChoicesIgnoringType method look like this:

public ViewletFactoryChoice [] determineFactoryChoicesIgnoringType(
        MeshObjectsToView toView )
{
    JeeMeshObjectsToView realToView = (JeeMeshObjectsToView) toView;

    ArrayList<ViewletFactoryChoice> ret = new ArrayList<ViewletFactoryChoice>();

    MeshObject subject = toView.getSubject();
    if( subject.getMeshBase().getHomeObject() == subject ) {
        ret.add( DefaultJspViewlet.choice(     realToView, “org.infogrid.myhealth.viewlet.patients.PatientsCollectionViewlet”, ViewletFactoryChoice.PERFECT_MATCH_QUALITY ));
        ret.add( AllMeshObjectsViewlet.choice( realToView, ViewletFactoryChoice.GOOD_MATCH_QUALITY ));
        ret.add( AllMeshTypesViewlet.choice(   realToView, ViewletFactoryChoice.AVERAGE_MATCH_QUALITY ));
    }

    ret.add( DefaultJspViewlet.choice( realToView, “org.infogrid.jee.viewlet.propertysheet.PropertySheetViewlet”, ViewletFactoryChoice.AVERAGE_MATCH_QUALITY ));

    return ArrayHelper.copyIntoNewArray( ret, ViewletFactoryChoice.class );
}

This means that preferably, the app’s front page is now going to be run by a Viewlet called org.infogrid.myhealth.viewlet.patients.PatientsCollectionViewlet. Which we’ll create by creating

• a JSP file called PatientsCollectionViewlet.jsp at location web/v/org/infogrid/myhealth/viewlet/patients/. For now, just put in a placeholder HTML content.
• a corresponding CSS file at web/v/org/infogrid/myhealth/viewlet/patients/PatientsCollectionViewlet.css.
• A property file that contains the name of the viewlet, at java/org/infogrid/myhealth/viewlet/patients/PatientsCollectionViewlet.properties.

(You can find their content in SVN).

Finally, we use the InfoGrid JSPO mechanism (”Java Server Pages Overlay”) to quickly add an editable GUI. The essence of that is this code in the JSP:

    <u:callJspo page="/o/personcollection/create-patient.jspo" action="${Viewlet.postUrl}" linkTitle="Create a new patient" submitLabel="Create">
     <img src="${CONTEXT}/s/images/add.png" alt="[add]” />
    </u:callJspo>

This includes a JSP fragment with the overlay dialog, which looks like this:

<h2>Create patient</h2>
<input name="shell.patientcollection.access" type="hidden" value="find" />
<input name="shell.patientcollection" type="hidden" value="users" />
<input name="shell.patientcollection.to.patient.perform" type="hidden" value="relateifneeded" />
<input name="shell.patientcollection.to.patient.blessRole" type="hidden" value="org.infogrid.model.MyHealth/PersonCollection_Collects_Person-S" />
<input name="shell.patient.access" type="hidden" value="create" />
<input name="shell.patient.bless" type="hidden" value="org.infogrid.model.MyHealth/Person" />
<table class="form" border="0">
<tbody>
<tr>
<th class="label">At URL:</th>
<td>
<input class="identifier" name="shell.patient" size="32" value="users/" /></td>
</tr>
<tr>
<th class="label">First name:</th>
<td></td>
</tr>
<tr>
<th class="label">Last name:</th>
<td></td>
</tr>
</tbody></table>

This uses the HTTP Shell to create a new MeshObject of type Person, which is related to the PersonCollection of users we created earlier, using RelationshipType PersonCollection_Collects_Person. Note that the .jspo file type needs to be declared as JSP file type in the app’s web.xml file.

A few additions to CSS and Javascript (again found in SVN), and we recompile and run.

Now we can add users to the app. Check it out!

MyHealth is supposed to be for individuals and their families. So we need a way to distinguish whose data is whose. To do that, we create a (very simple) model:

In the model project (org.infogrid.model.MyHealth), we open model.xml and add two EntityTypes and a RelationshipType:

• Person — represents a person, such as myself, or perhaps later also doctors. For now, it only has FirstName and LastName properties
• PersonCollection — a collection of people. These collection objects turn out to be rather useful when representing information as a graph, as we do with InfoGrid
• PersonCollection_Collects_Person — lets us collect Persons in PersonCollections. We make the multiplicity 0:N/0:N to enable the same person to be part of multiple collections. Not sure we’ll need that later but it won’t hurt,.

Then, compile in NetBeans. This automatically runs the InfoGrid code generator, which generates Java interfaces and classes for Person and PersonCollection etc.

Back to the web project org.infogrid.myhealth.www. So far, we hadn’t added a dependency to the model project, which is of course needed so that the web app can use the model we just created. This means we need to add the dependency in the NetBeans project property dialog, and also in the project’s module.adv file.

NetBeans has the unfortunate habit of sometimes getting confused about directory paths; a quick check to project.properties to make sure that hasn’t happened.

Now we’d always like to have an instance of PersonCollection around that acts as the directory of people managed by MyHealth. One good way of doing this is to pick a fixed identifier (say “users”) for the PersonCollection, and to instantiate it when the app starts.

To do that, we open class AppInitializationFilter in the web project, and override method populateMeshBase, which exists for that purpose. In that method, we do this:

mb.executeAsap( new TransactionAction<Void>() {
    public Void execute() throws Throwable {
        MeshObjectIdentifier userCollectionId = idFact.fromExternalForm( "users" );
        MeshObject           userCollection   = mb.accessLocally( userCollectionId );

        if( userCollection == null ) {
            life.createMeshObject( userCollectionId, MyHealthSubjectArea.PERSONCOLLECTION );
        }
        return null;
    }
} );

In other words, if it does not exist yet, we create a MeshObject of type PersonCollection (the type we defined above) with identifier “users”.

Time to compile and run.

Now the MeshBase contains two MeshObjects: the default home object, and the object of type PersonCollection that we just defined at identifier “users”.

Good enough for today; next time, we’ll create some HTML that allows us to create and edit users. As usual, you find today’s code in SVN at http://svn.infogrid.org/infogrid/trunk/ig-app-myhealth.

In the first installment, we created an executable, but feature-less application scaffold for MyHealth. Now we’ll modify the basic layout for the app so we can add features in the right places next.

This is easiest if we first run MyHealth under Tomcat right from NetBeans: It still uses the layout from MeshWorld, and says “Unnamed App”. To change that:

• we open the JSP template that controls the main layout of the app, which is at web/s/templates/default/text/html/template.jsp. As one can tell easily from this path, InfoGrid also supports non-default templates with file formats other than text/html, but for now, we won’t use those features.
• let’s remove the big InfoGrid icon to the right; built on InfoGrid at the bottom is good enough
• the app’s main icon to the left needs replacing, so I just whip something up real quick in Inkscape, save it to web/s/images/myhealth.png and change the path in the template file
• we can leave the footer for now; I don’t know yet whether we’ll use the famfamfam.com icons again or not, let’s leave them in for the time being.
• now for the CSS, remove some of the lines in color.css and adjust some CSS elements in layout.css

That’s all for today. Code in SVN as usual.

This is the first of a series of posts documenting the creation of an open-source web app called MyHealth to manage personal health and wellness data, based on the InfoGrid Web Graph Database.

This scratches a personal itch, and I figure I might as well chronicle the progress of this little side project so developers new to InfoGrid get a taste.

I don’t think I’m the only geek who’d like to manage their health, wellness and fitness information better and keep it under their own control, so perhaps you might want to help out. There seems to be little or no open-source software for personal health information management that was written in this century, or so, and it will be cool to do it based on cutting-edge NoSQL graph database technology.

I’ll check today’s code every day and post on this blog. As it is a side project, I don’t expect to do more than an hour of work each day, and likely less. You are very welcome to contribute if you like, just send a note to the InfoGrid mailing list.

So here is day 1:

1. Let’s get the InfoGrid code:

mkdir -p ~/svn/svn.infogrid.org/infogrid
svn checkout http://svn.infogrid.org/infogrid/trunk

2. Make sure our setup is right and InfoGrid builds

cd ~/svn/svn.infogrid.org/infogrid/trunk
ig-tools/build.sh -a

This takes a while, so off to writing some of this blog post.

Oops, test failed because I forgot to start Tomcat. So let’s run NetBeans, which I’ll use as my IDE, and run Tomcat from there. Then re-run the last of the user interface tests that depend on Tomcat:

ig-tools/build.sh -run ig-ui

3. We’ll going to put MyHealth right into the InfoGrid SVN:

mkdir ig-app-myhealth
mkdir ig-app-myhealth/modules
mkdir ig-app-myhealth/app

I learned this past week that this directory structure is best described as an “InfoGrid-ism” (Hi Ben!) i.e. a convention that makes using InfoGrid easier. The modules subdirectory will contain the modules from which MyHealth is assembled, other than the InfoGrid modules, and the app directory will contain the web app project.

4. Use the brand-new project creation scripts:

cd ig-app-myhealth/modules
../../ig-tools/create-model-module.pl --name org.infogrid.model.MyHealth \
    --infogridpath ../..

This created a ModelModule, aka a schema for the graph database. Which also can be opened up as a project in NetBeans.

Then we also create a web project by cloning the MeshWorld example app:

cd ../app
../../ig-tools/clone-meshworld.pl --name org.infogrid.myhealth.www \
    --infogridpath ../..

5. Open both of those projects in NetBeans to make sure everything is right and can be built.

Select “Clean and Build” first on the model project, then on the web project. Works fine. We haven’t told those project about each other, but we will later.

6. Setting up a MySQL database.

In MySql, create a database and set some privileges:

mysql -u root -p
mysql> create database myhealth;
mysql> grant all privileges on myhealth.* to 'myhealth'@'localhost' identified by 'secret';
mysql> flush privileges;

In the org.infogrid.myhealth.www project in NetBeans, edit the context.xml file to read as follows:

<?xml version="1.0" encoding="UTF-8"?>
<Context path="/org.infogrid.myhealth.www" cookies="false">
  <Resource auth="Container"
            type="javax.sql.DataSource"
            driverClassName="com.mysql.jdbc.Driver"
            name="jdbc/org.infogrid.myhealth.www"
            url="jdbc:mysql://localhost:3306/myhealth"
            username="myhealth"
            password="secret"
            maxActive="20"
            maxIdle="10"
            maxWait="-1" />
</Context>

To stay on the safe side, rebuild the web project.

6. Assigning an App Server and Run

In NetBeans, open the properties for project org.infogrid.myhealth.www, select the Run tab, and pick a local Tomcat installation as the assigned app server.

Running the web project. It runs! It is totally and utterly useless so far, and has no health content or health-relevant schema of any kind. But it runs, and we’ll keep the MyHealth app running from now on regardless how many changes we make and features we write.

7. Check-in so you guys can check out what has been created so far:

cd ../..
svn add -N ig-app-myhealth ig-app-myhealth/modules ig-app-myhealth/app
ig-tools/add-project-to-svn.pl ig-app-myhealth-modules/org.infogrid.model.MyHealth
ig-tools/add-project-to-svn.pl ig-app-myhealth-app/org.infogrid.myhealth.www

(These Perl scripts are just wrappers around svn to to check the right files and directories of NetBeans projects into SVN)

Done for today. We have a working app even if it does not do anything useful yet.

And, as JP is fond to say, “so to bed”.

It’s been a while since the last release, but why change if users are been generally happy with it … Here’s what’s new:

Highlights:

• Native support for currency values (e.g. $1.23 USD)
• Even simpler JSON generation
• StringDataTypes can now carry a regular expression, which is checked before assignment (e.g. makes it impossible to assign a String that isn’t to a property that must contain an e-mail address)
• Support for cascading delete
• InfoGrid compilation is now reported to work on Windows
• Multi-tenancy support for LID
• More flexible handling of HTTP status codes in Probe framework (e.g. what should happen upon a redirect)
• Extensions to custom JEE tags and HttpShell to make JSP programming with InfoGrid even simpler, e.g. simple tags to create Javascript-based overlay forms
• Improved error reporting
• Additional tests
• Lots of small usability improvements and bug fixes

Here are the gory details:

Core:

• ModuleAdvertisementSerializer to generate relative paths to JARs
• more types of MeshObjectSelector
• added getLocalId to NetMeshObjectIdentifier
• added guessFromExternalForm to IdentifierFactory
• use “” not null for localId of home object
• Augmented set of pre-defined BlobDataTypes
• Improved BlobDataType to check for MIME compatibility with regexes
• added CurrencyValue and CurrencyDataType to represent money as a primitive type
• added convenience method to MeshObject: determine whether the relationship to a neighbor MeshObject is blessed with a certain RoleType
• Moved OnDemandTransaction from HttpShell into utils
• implemented cascading delete support
• added TimeStampValue.earlierOf and TimeStampValue.laterOf for convenience
• Special value NOW for TimeStampValue parsing
• regex can be be specified for StringDataTypes
• added AbstractDelegatingLocalizedRuntimeException for symmetry and completeness
• better error reporting for TransactionActionExceptions
• Explicitly set latin1 character set in MySQL store to avoid key size error when the default character set is multi-byte.
• TransactionAction.execute() now does not take a Transaction parameter any more, but provides it as a member variable
• check raw id even when guessing MeshObjectIdentifier
• when there’s a ParseException during parsing of store content, don’t return null but throw NoSuchElementException
• Added MeshObjectHelper for simpler JSP programming with TypedMeshObjectFacades
• Carry specify exceptions keyed by failed MeshObjectIdentifiers in MeshObjectAccessException, instead of swallowing all exceptions other than the first
• MeshObjectsNotFoundException is not a subclass of MeshObjectAccessException any more
• Correct singular/plural error messages for MeshObjectAccessException and MeshObjectsNotFoundException
• NetMeshObjectAccessException now carries redirect NetMeshObjectAccessSpecifications
• NetMeshObjectAccessSpecification is now an Identifier
• Migrated sweeping functionality from the various MeshBase implementations into Sweeper. The policy functionality of Sweeper became SweepPolicy.
• Better pingpong debugging support
• created MeshObjectSet.hasSameContent
• upgrade to warn if MySQL database cannot be detected/created
• Id column in MySQL needs to be case-sensitive
• don’t report exception in rollback that isn’t an error
• default expiration for UnnecessaryReplicasSweepPolicy
• Allow the sweep of a single MeshObject
• MeshObject.delete and sub-methods have no business to throw NotPermittedException
• Non-semantic “kill” of a ShadowMeshBase
• Kill button in ShadowAwareAllMeshBasesViewlet
• No need for public method initiateCeaseCommunications on Proxy
• Don’t try to talk to Proxies when NetMeshBase is dead
• Tolerate some IsDeadExceptions when dead (duh)
• Use default mime type for BlobValue when none is given when parsing StringRepresentation
• Avoid that IsDeadException kills meaningful error message
• support typefield XML attribute for StringDataTypes as well
• added StringDataType for e-mail
• fixed automatic merge problem
• allow MeshObject.equals( TypedMeshObjectFacade ) for simplicity
• use get_XXX pattern in TypedMeshObjectFacade for non-Properties/non-pseudo-Properties to avoid name clashes with code-generated setters
• use get_XXX pattern in TypedMeshObjectFacade for non-Properties/non-pseudo-Properties to avoid name clashes with code-generated setters
• renamed Rfc3339Util to DateTimeUtil and expanded to W3C timestamps
• DefaultNetMeshBaseIdentifierFactory now uses pluggable “Scheme”s to support identifiers of various protocols, including http, https, file, xri, acct
• renamed UnknownProtocolParseException to UnknownSchemeParseException
• added ability to specify Maps in Resource files
• When parsing NetMeshBaseIdentifiers, turn each String into canonical form as well, e.g. hTTp: and http:
• added toColloquialExternalForm() to Identifier; more sane
• removed IdentifierStringifier.toColloquialIdentifier()
• use guessFromExternalForm in more places for more tolerance (e.g. when reading from Store to be more tolerant of Scheme changes)
• be more tolerant of Exceptions when restarting MeshBases; try best-effort
• Better logging of start / die of InfoGridWebApps
• when resetting PingPong, only increment token by 1
• more defensive programming if Storage layer is corrupt
• refactor message sending framework to always deliver all received messages together instead of singly
• merge two XprisoMessages before processing if possible
• added ByPropertyValueSelector and PropertyComparisonOperator
• NotRelatedException not thrown any more when testing whether two MeshObjects are related by a given RoleType; was counter-intuitive and pedantic
• StringRepresentationParameters must always be given.
• Make it easier to generate MeshObjectIdentifiers with different lengths
• upgraded Postgresql JDBC for 64bit support
• Netbeans 7 support
• Make sure Threadpool is always cleanly shut down. Now Tomcat won’t hang occasionally after undeploy.
• support custom regex violation error messages for PropertyTypes that specify regular expressions
• refactored L10Map into L10PropertyValueMap and L10StringMap, moved to utils package
• more HTML class declarations in BlobValue HTML
• added invert method to most MeshObjectSelectors
• added MeshObjectSet.subset convenience method
• Don’t include dead MeshObjects in MeshObjectSets
• factored out ExternalCommand so invoking an external process becomes simpler
• default MeshObjectIdentifier to contain _ instead of ~

Identity-related:

• AccountManager needs a site parameter for determining account to support multi-tenancy
• generate login event even if only the session gets renewed
• XrdsProbe to cache the XRDS file
• added content and content type as parameters to XmlProbe interface; enables XrdsProbe to cache the XRDS file
• added getGroupNames to LidAccount
• check group membership by group identifier as well as group name
• also check group membership by groups set as request attribute
• factored out some symbolic OpenID parameter names
• fix when associations expire
• Better integrated AccessManager and ThreadIdentityManager
• root may do anything in AclBasedAccessManager
• Split AclBasedSecurity into two: an ACL-based AccessManager implementation that does not rely on Guards, and an AccessManager implementation that evaluates Guards but is independent on a particular model (such as the AclBasedSecurity model)
• Eliminated org.infogrid.model.AclBasedSecurity, and instead introduced org.infogrid.meshbase.security.aclbased, which also includes a model
• DelegatingAccessManager now supports delegating to multiple other AccessManagers, all of which must agree
• added ThreadIdentityManager.suExec
• When there isn’t a ProtectionDomain, it’s free for all in AclBasedAccessManager
• moved org.infogrid.model.SecurityTest into the attic; currently not used
• added expiring credentials to AuthenticationStatus
• expired OpenID associations throw OpenIdAssociationExpiredException
• OpenID dumb mode fixes
• Added getCarriedExpiredCredentialTypes to LidClientAuthenticationStatus
• added org.infogrid.lid.model.account/Account_LastLoggedIn which can come in handy
• Removed Account Status value CREATED — not particularly useful and would have needed update logic
• credential values are now separate from LidAccount, e.g. the StoreLidPasswordCredentialType knows where to look for the password
• don’t need LidCredentialType.isRemote any more
• allow password change from StoreLidPasswordCredentialType
• removed realm from LID; never quite used
• better support for multi-tenancy / virtual hosts in LID: carry siteIdentifier around in more places
• distinguish one-time-tokens from other types of credentials in LID
• more methods for setting, checking, resetting password credentials
• support that MeshObject can own itself in AclUtils
• A little helper utility to generate secret keys

Model-related:

• Code-generate an entry for the SubjectArea itself, too
• exactly one SubjectArea element is expected in each model element in the model XML
• Added HTTP status code and location properties to WebResource
• minor rename of SubjectArea user-visible name

Probe-related:

• Handle HTTP redirects better when encountered by Probes; existing behavior should be unchanged however
• Do not throw FactoryException as a direct cause of NetMeshObjectAccessException but its cause instead
• Introduced ProbeException.HttpRedirectResponse
• Added NetMeshBaseRedirectException, so ProbeException.HttpRedirectResponse does not need to be moved up to module org.infogrid.kernel.net
• Introduced HttpMappingPolicy that enables different policies for mapping HTTP non-OK status codes into InfoGrid; default remains the same behavior
• Change in API to set up ProbeManagers etc.: ProbeDirectory passed to ProbeManager instead of ShadowMeshBaseFactory, then ShadowMeshBaseFactory is notified where ProbeManager can be found
• ProbeException.HttpErrorResponse not needed any more
• Support HTTP redirects that give relative URLs as Location, against the HTTP spec (I’m looking at you, Google)
• HttpMappingPolicy has moved from ProbeManager to ProbeDirectory
• Fixes issue where wrong HttpMappingPolicy was set upon restore of Shadow from disk
• Throw exception if ProbeManager is configured without a ProbeDirectory
• added basic XRD support to InfoGrid
• added acct: as one of the basic protocols supported by NetMeshBaseIdentifier
• added Webfinger support
• Don’t throw ProbeException.DontHaveNonXmlStreamProbe if the found content is of length zero

Viewlet/GUI-related:

• Renamed NotIfViewletState to NotIfViewletStateTag for consistency
• throw correct error message if no subject
• fix status in AbstractSetIterateTag
• colloquial=true by default
• by default, sort alphabetically in TreeIterateTag
• fix display of <HOME> for home object broken in last checkin
• added potentiallyShorten to WebContextAwareMeshObjectIdentifierStringifier
• instrument to track HTTP client-side calls
• trim strings first to deal with sloppy JSP writing
• added request property to JeeMeshObjectsToView
• change default app server to Tomcat6
• pass SaneUrl through to ViewletFactory, so it can access request attributes
• created org.infogrid.jee.security.aclbased tag library for AclBasedSecurity Subject Area
• added TextStructuredResponseSection.containsContent
• replaced ViewletAlternativesTag.js with more general-purpose ToggleCssClass.js
• support redirect to newly created object in HttpShell
• Created BracketTags, which allow conditional generation of, say, <ul> and </ul> tags depending on whether or not the content tag has any non-whitespace content.
• factored out AbstractSaneRequest.urlWithoutMatchingArguments for easier reusability
• trim entered identifiers before trying to resolve them in HttpShellFilter
• Better error reporting for HttpShell
• IncludeViewletTag more robust if path not specified
• Removed the List<Throwable> in request attribute for collecting processing exceptions; now abstracted into new interface ProblemReporter
• Allow null identifiers for create access verb in HttpShell again
• hyperlink on UnsafePostException’s message
• TitleTag to use a separate section in the StructuredResponse
• default mechanism for TitleTag with Viewlet name and app name, can be overridden with TitleTag
• <tmpl:title> tag prints <title> tags itself
• fixed userVisibleName on Viewlet
• JeeViewlets’ default POST URL used to sometimes leave out reached MeshObjects, which would list all found-by-traversal MeshObjects after post, which was very confusing. Removed traversal spec in case there are no reached MeshObjects
• Pass subject’s userVisibleString into default Viewlet title construction.
• Created InvalidViewletActionException.
• enable logging of when InfoGridWebApps start and stop
• added HttpShellHandler to HttpShell
• rollback Transaction if there is an exception during execution by HttpShell
• support for radiobox-driven “which MeshObject should I be related to” relate/unrelate from the HttpShell
• Added sweep and accessLocally to HttpShell HTML
• Created ProxyTag in analogy to emitting other objects like MeshObjects etc.
• Sweep support for the HttpShell
• Added limit parameter to SetIterateTags in analogy to SQL limit
• Added MeshObjectSetIterateBeyondLimitTag that gets invoked when a MeshObjectSet iteration stops due to a limit parameter so the JSP can display a “more” link or something like that
• display trailing slash on URLs even if isColloquial; if not, user cannot distinguish URLs with and without trailing slashes
• added IfNullTag and NotIfNull tag
• allow to set properties via the HttpShell at the same time the MeshObject is only being created
• working on allowing to set properties via the HttpShell at the same time the MeshObject is only being created
• added allowNull attribute to PropertyTag to remove “remove” label for optional properties when so needed
• fix ignore=”true” bug in MeshObjectTag
• implemented a “JSP subroutine” set of custom tags
• allow TypedMeshObjectFacade in place of any MeshObject for JSP custom tags; makes life simpler
• use ‘block’ instead of ‘inline’ for create label in PropertyTags, so CSS can be used to move label vertically
• added the just-completed Transactions to the invocation of HttpShellHandler, so implementations have a chance to understand what happened
• added deleted MeshObjects to variable map passed to HttpShellHandler
• tag to easily report a problem into the error log
• tags to safeguard JSPs against inadvertent invocation by the wrong user or with the group user groups
• support AccountStatus APPLIEDFOR and REFUSED
• change default HTML title if Viewlet reports error; that way we don’t accidentially leak information if there is an access control problem
• Re-implemented CSRF remedy: now match cookie and form instead of issuing form tokens
• use lowercase cookie names
• Added NotIfErrorsTag and NotIfInfoMessagesTag
• Use StringRepresentation framework to format the label String for null PropertyValues
• augmented HttpShellHandler to have invocable method before, after, at the beginning and at the end of Transactions
• MeshType may be given directly in PropertyIterateTag
• Change where <HOME> special identifier is defined. Stringifying MeshObjectIdentifier for HomeObject now produces “”, while stringifying MeshObject produces <HOME>
• HttpShell now recognizes “not set” value because “” is now a valid value for a MeshObjectIdentifier
• added JspoXXXTags
• removed now unnecessary org.infogrid.jee.store project
• carry incoming request in DefaultJeeNetMeshObjectsToView
• Allow both bean and identifiers in AbstractRelatedTag and subclasses
• always use multi-part encoding for Jspo forms so file uploads are always possible
• make JSP tags more consistent that use MeshObjects: can now specify MeshObject directly, MeshObjectIdentifier (direct and as String), or name of bean
• added MeshObjectBlessedByTag
• allow optional parameters on Jspo and Jspf tags
• Support $1.23 in addition to 1.23 USD for CurrencyValue
• support different capitalizations for MeshObjectBlessedByTag
• SingleMemberTraversalTag for convenience
• additional keywords for HttpShell
• allow HttpShell to override/specify direction of a redirect after shell execution
• additional callback afterAccess for HttpShell
• allow immediate activation of called JspoTag through property on call
• Added Json viewlet. New viewlet that will write a direct acyclic object graph in Json to level n.
• added custom tag that makes it easy to iterate over the intersection of two traversal sets
• allow MeshObjectBlessedByTag to write to a variable instead of to stdout
• rollback transaction when HttpShellHandler throws HttpShellException
• added MissingArgumentException that turns out to be generally useful
• Created custom tags to help matching URL arguments easily
• fixed incorrect processing of RuntimeException in HttpShell (particularly from HttpShellHandlers)
• DataType would sometimes invalidly truncate HTML output with maxLength specified. Hope this implementation is better.
• slight refactoring of JeeViewlet-internal API to make it easier to remove URL parameters from Viewlet’s default Post URL

MeshWorld/NetMeshWorld-related:

• Show number of MeshObjects in the MeshBase in AllMeshObjectsViewlet
• round borders for webkit-based browsers too
• NetMeshWorld has no business depending on AclBasedSecurity at this time
• introduced property AnetMeshBase.ALLOW_NON_LOCAL_MESHOBJECT_CREATION to allow non-local NetMeshObject creation — useful for test instrumentation, for example
• created ModuleInventoryViewlet
• added app resource file to MeshWorld and NetMeshWorld
• Added Sweeping button into NetMeshWorld.
• Added ability to filter MeshBase content in AllMeshObjectsViewlet
• Enable AllMeshObjectsViewlet to view empty sets.
• Added properties file for Log4jConfigurationViewlet
• add direct link to PropertySheetViewlet from AllMeshObjectsViewlet
• Added filtering by home proxy to AllNetMeshObjectsViewlet
• NetMeshWorld uses AllNetMeshObjectsViewlet, not AllMeshObjectsViewlet now
• added “requires Javascript” via <noscript> to templates in example and test apps
• Sort EntityTypes alphabetically in AllMeshObjectsViewlet

A new viewlet was recently added to the build called JsonViwelet that renders a graph of objects in Javascript Object Notation (Json). It’s packaged in a new module called org.infogrid.jee.viewlet.json to isolate its dependency on “Jackson” - an open source Json parser, from the rest of the ig-ui modules, which is checked in to the ig-vendors package.

This viewlet was provided to help you write an Ajax user interface. Of course, it does not solve the whole problem of creating a modern browser experience which includes visually rendering objects in a browser window, and synchronizing them with the server running Infogrid. However it does kick-start the process and is one less issue to deal with.

There is already support for Json in the tree, from a Viewlet in the ig-ui MeshWorld test apps. called ObjectSetViewlet.jsp. As its name suggests it is a java server page implementation. It’s packaged conventionally in the web tree in v/viewlet/objectset/ObjectSetViewlet/text/json so that it can be dispatched when you provide a mime type of “text/json” in the lid-format parameter of the request URL. As its name suggests it’s dispatched on a collection of objects returned by traversing from the object identified in the request URL - e.g. http://localhost:8084/org.infogrid.meshworld.net/nnnnn?lid-format=viewlet:org.infogrid.jee.viewlet.objectset.ObjectSetViewlet&lid-format=mime:text/json&lid-traversal=org.foo.Test/Entity_Collects_AnotherEntity-S

This serves as a useful example for how to develop your own Json viewlets, however you could soon end up with several special purpose jsp viewlets that could say render the specific object in the request as Json, render it with and without its meta information and so on … These different rendering requirements are based on how you intend to use the objects into which the Json response gets transformed when it’s evaluated in the browser.

So it seemed like a better idea to provide a universal viewlet that could fulfill all these requirements by being passed URL arguments to control its behavior. It works by performing a transitive closure on the object graph from the object specified in the URL - without looping. Whenever a cycle is detected it writes the object’s “id”  but not its contents, to stub the graph so that the client has a reference it can deal with later.

Unless constrained by the arguments, its default behavior is to dump everything about every object that is encountered in the traversal. So be warned - if you have a very interconnected graph you could end up dumping the whole contests of your meshbase.

The JsonViewlet takes the URL arguments:

• level=n|0 - 0 means no limit and will follow every relationship
• dateenc=func|string - determines how Date is rendered, either as a function or String
• ignoreblessing=entitytype_name_regex - names of blessings to be ignored
• meta=[no|false]|[true|yes]|only - whether or not to dump meta information
• trimsa=[yes|true]|[no|false] - trims the subject area name from the property names

Bold = default.

Level, determines the depth of the traversal. Since the object identifiers are returned for the next lower level you could provide code in your javascript client to later make another call to return the next levels in the graph.

Dateenc, controls how IG TimeStamp’s are rendered in json. Json does not provide a way to encode dates, so there is a choice to do this either as a string or as a javascript Date constructor method signature. They both work - you choose.

Ignoreblessing, has the effect of not dumping the values of the specified blessing. This is primarily useful for removing the Probe model properties which bless probed entities. This is control information that is probably not needed by a client - unless you are say writing a probe manager. You can pass it a regular expression or the fully qualified class name, and the argument can be repeated or comma-delimited to specify as many of these as you like.

Meta, gives you the option of either supressing the meta information or only dumping the meta information about an object. Javascript is a typeless language and probably does not care too much about the meta information. However there are some client frameworks that are wrestling with the issue of java-javascript mapping and doing things like simulating fixed class hierarchies - so maybe the meta information might be important.

Finally there is trmsa - which simply trims the subject area name off the front of property type names. These strings can get a bit long and often they are not needed if you’ve specified the property type name uniquely.

There is some error checking of these arguments, and if any error is encountered a Json error is returned as an “id” and “msg.” There should be no other errors returned.

Json returns values as either numbers or strings, therefore blobs are treated as references and returned as URL’s that reference the specific property, and request the PropertyViewlet do the rendering.

In order to use this viewlet you’ll need to make sure it’s added to the application’s viewlet factory, using something like this:

ret.add(JsonViewlet.choice(realToView, ViewletFactoryChoice.BAD_MATCH_QUALITY));

So please give it a try. It would be nice to build out more components needed to build a rich client experience.

Alex Popescu argues that document databases have an impedance mismatch with object-based software. He compares the situation to relational databases:

“One of the most often mentioned issues reported by software engineers working with relational databases from object-oriented languages is the object-relational impedance mismatch. Document databases adopters are saying that one benefit of document stores is that there is no impedance mismatch between the object and document worlds.

“I don’t think this is entirely true.”

Given that databases and in-memory representations of objects serve different purposes, I don’t think we’ll ever completely manage to do without mappers of some kind. (Otherwise we’d all be using Java Object Serialization as our database.)

However, to me at least it appears that of all database technologies, graph databases like InfoGrid have the smallest impedance mismatch: a graph of objects is fundamentally closer to what is happening in memory of an object-based application than any other representation (e.g. document-based / hierarchical / relational / key-value etc.)

InfoGrid trunk has been upgraded to NetBeans7, but for the time being, we remain with Tomcat6. Unfortunately the NetBeans guys have not exactly made it easy to remain compiling JSPs against the Tomcat6-supported specs instead of Glassfish.

In the relevant InfoGrid projects, we set the relevant properties in project.properties for Tomcat6. They are picked up when running the build from the command line.

If you find yourself with compilation errors when compiling from the IDE, the following trick might help:

• Find your system-wide NetBeans 7.0 properties default file, e.g. ~/.netbeans/7.0/build.properties
• In that file, find the properties that start with libs.jsp- such as libs.jsp-compiler.classpath.
• Comment them all out!

Given the sequence in which NetBeans reads in those property files (system-wide, private project-specific, all project-specific), they might override the correct settings otherwise.

The recent workshop on Graph Databases in Barcelona sparked an interesting debate among vendors of graph databases about how to accelerate graph database adoption.

I don’t think that debate has been resolved yet. Perhaps this post will help a bit.

Some opinions that I heard were:

• there are significant differences between the various graph database products on the market (e.g. graphs, property graphs, properties on edges or not, hypergraphs etc.). Unless they are more similar, customers will fear lock-in and not buy any of them. Here is a variation:
• relational databases only took off once there was agreement on the SQL standard among vendors. We need to cooperate to create a similar language, otherwise graph database adoption will not take off either.
• most potential users of graph databases have never heard of them. How could they use any if they don’t know they exist?
• even if possible users know of graph databases, they do not know what the use cases are because use cases and success stories have not broadly been documented.

What do you think the biggest obstacles are for graph database adoption?

Graph databases like InfoGrid are much better at traversing recursive data structures than relational databases. That is probably an established fact by now and not controversial.

But what does this mean in practice?

Here is an example just how simple it can be to generate a hierarchical HTML outline from a graph using InfoGrid’s custom tags:

<dl>
  <tree:treeIterate
            startObjectName="Subject"
            traversalSpecification="xpath:*"
            meshObjectLoopVar="current">
    <tree:down>
      <dd><dl>
    </tree:down>
    <tree:up>
      </dl></dd>
    </tree:up>
    <tree:nodeBefore>
      <dt>
        <mesh:meshObject meshObjectName="current"/>
      </dt>
    </tree:nodeBefore>
  </tree:treeIterate>
</dl>

The result is a set of nested HTML definition lists, which are a good match for an outline in HTML.

By way of explanation of this example, the outer tag is a big loop, which iterates over all elements in the outline. Every time we walk down the tree, the <tree:down> tag fires and emits the HTML list start element contained there. Every time we walk up the tree, the corresponding <tree:up> tag triggers and closes the list. “Subject” is the name of the start node as JSP bean, and the xpath-like expression is used to describe which branches to traverse in the graph, because it would just as simple traversing the opposite direction, or sideways.

I don’t think it gets much simpler Imagine the pain in SQL, particularly if the tree is deep …