Thursday, March 20, 2008

ViSit Anywhere Architecture

This article provides some insights into the ViSit Anywhere architecture, and shows exactly why we think that it can be used to as a foundation for work-centered GIS and geo-engineering applications. Physically, the architecture of ViSit Anywhere is straight forward. The following diagram summarizes the basic components.

I have added several components here to represent the different roles that can be taken by the various applications and users. Not all components need to reside on physically distinct machines.

The first thing to note is that this is a client server architecture. The consolidation point for ViSit Anywhere is the Project Server. This is a component that maintains the stack of changes that have been applied to a the projects used in ViSit Anywhere. An installation can consist of any number of projects. When creating a new project, the project server will simply deliver a project consisting of the basic ViSit Anywhere types, but without any schema or instance data.

Internally, the project server is implemented as a .NET WCF server and is delivered either as a HTTP web service or a Windows service that listens to clients on a TCP/IP port or both. In fact, the two configurations of the project service can be active at the same time, as some users may find it more convenient to connect to the project data through ubiquitous HTTP, while others may prefer the performance of a dedicated TCP/IP port. Switching between the two network schemes requires only a reconfiguration of the client.

The project server acts as a consolidation point for ViSit Anywhere licensing as well. When users connect to the project server their user credentials are sent to the license server where they are tested against the available block of licenses. In this way licensing has been localized on the server and the GIS administrator does not have to micro-manage installing licenses on individual machines. The licensing model has been design specifically for disconnected editing, where the synchronization and commit processes act data integration portals for the users.

I will come back to the role of the database server later in this blog. For now, I want to talk about the ViSit Anywhere clients. I have represented the clients with three separate icons. They represent an integration client, a number of standard workstations and a number of mobile workstations. Each of these roles is important for ViSit Anywhere usage.

The first client role depicted is the integration workstation. In this case, data from various sources and in various formats are integrated into the ViSit Anywhere project. ViSit Anywhere uses a simply abstract data format, based on XML to represent all underlying data items. Each component renders the underlying data in a format that is appropriate for its implementation. For example, a Microstation-based editing client maintains data in the generic format, but transforms it into a Microstation design file when it needs to edit or display it. The format used by the standalone editor does not maintain data in Microstation design files and these editors do not have to know how to display a Microstation design file.

The integration workstation will typically use Microstation as the primary CAD engine, as Microstation is well adapted for reading and converting various graphic formats. Also the general CAD editing tools found in Microstation are useful for creating symbols and other graphic items that will later be used by the other CAD engines.

The intergration workstation may also need to extract data from a ViSit v3 project using an RDBMS during project initialization. Project migration from ViSit v3 is performed by running a program that creates an XML description of the ViSit project, then building the project schema and instance data by "discovering" the described information in ViSit Anywhere. The migration process requires a connection to the RDBMS which may require a client database license. Once the project has been migrated and integrated into the ViSit Anywhere project, the database connection infrastructure on the client is no longer necessary. For this reason, we believe it necessary to have only a limited number of integration workstations.

The next client role is that of general workstation. This represents a typical in-office user. These users could be outfitted with either the ViSit Anywhere Microstation or ViSit Anywhere standalone. Microstation is typically only necessary when the structure of the data will require the higher performance offered by this tool. For example, Microstation is able to better optimize the display of large raster files. Even if Microstation is available (for example, because it is used for non-ViSit Anywhere work flows), users may prefer using the standalone version because of its simplified user interface. ViSit Anywhere licensing allows users to have both the standalone or Microstation version.

Standard workstation users will have the ability to modify the GIS data like all other user. They do not require a connection to a database or any other 3rd party software (including database clients and CAD editors). When these users are ready to commit their changes they can simply connect to the project server (via TCP/IP if the project server is on the LAN) and share their data. One interesting possibility that is presented by the ViSit Anywhere notion of "local project data", is to have users perform simulations that will not necessarily ever be committed. This is done by synchronizing to the tip of the project and making proposed changes. The user can then analyze the result of the changes and repeat the process if necessary, without fear of ever corrupting the project data. In addition, once users find configurations that are viable, they can be merged in to the project data simply by synchronizing to the tip of the project data, and committing their changes.

The final client role is the mobile worker. These users are typically equiped with a rugged laptop or tablet computer. They will typically synchronize the project on their mobile machine when they have the opporatunity. Their connection to the project server may only be sporadic and thus it is important that their data merges correctly and easily with the ever changing project data. ViSit Anywhere provides support for this through its change management system. Under this system the graphic and attribute data of managed elements are handled atomically. Since GIS data typically represents large sets of managed data, it is rare when two users modify the same element at the same time. That is, it is rare when there is a true conflict. Other, structural conflicts (like the key conflicts that occurs when two users both add a new element and the new elements both have the same key) can be resolved automatically by the system and do not require any user intervention. In ViSit Anywhere conflicts are resolved on the client's machine. There is not reason to upload revisions to the server or to use special tools to merge revisions. When users resolve conflicts the server provides the current server version and the last shared ancestor version to compare with the local version. The user simply modifies the local project so the data is correct then accepts the conflict as being resolved. The correct version is then accepted as the tip of the project when the changes are committed. I will talk more about conflicts and change management in an upcoming blog entry.

Note that on the above diagram all the connections between the client applications and the project server are shown in green. This is meant to represent the fact that these users can work productively without any connection to any other machine. The users only have to connect to share there changes with other users (by committing the changes) or to synchronize their local project to see the data other people have changed. In addition, no other 3rd party software is required to work productively while disconnected. This is equally true for disconnected mobile workers as it is for office workers.

The final component on this diagram is a server-side client that listens for project data changes and automatically updates an external RDBMS. The connection between the project server is shown in gray because this component is only scheduled for release in the second ViSit Anywhere beta. We are currently planning to implement auto-synch database clients for Oracle locator and Microsoft SQL Server 2008 (which also supports spatial data). We feel that These types of clients are very important as they will allow GIS project data to be easily accessible in other enterprise applications.

This concludes my basic look at the ViSit Anywhere architecture. The thing to understand is that ViSit Anywhere uses a centralized project server to consolidate and distribute data to all users, but once a user has synchronized to the server, that user can work autonomously, without further interaction with any external application until such as time as they want to share their changes or recover a newer version of the project data.

Wednesday, March 12, 2008

The ViSit Anywhere Data Model

Before we can really understand the novel features of ViSit Anywhere, it is important to understand its data model. When we were designing ViSit Anywhere, we started with the ViSit site model. From our point of view, the ViSit site model had proved its utility, as it had long provided a flexible, high performance foundation for our Microstation applications. ViSit Anywhere exploits the fundamental ideas that were present in ViSit, but it generalizes them and applies them over the various information layers used in the application.

Before I start with this discussion we should understand the three fundamental layers of the ViSit Anywhere data model, concrete data, schema data and type data. These layers exist in most application, but are rarely treated as explicitly as they are in ViSit Anywhere. At the highest, most concrete level we have the engineering project data. This is a model of an engineering asset, like a water or electricity network. The elements in this layer, the pipes, electric lines, equipment and serviced customers - all have concrete representations in the real world. This is the level of data that makes the system pay. The more closely this layer of data represents real world objects, the more utility you can extract from your system.

At the next layer in the data model we have what we like to call the schema data. This level can be thought of as an organized description of the concrete model. A well developed schema layer will make accessing and updating the (large) concrete data layer much easier. Many GIS applications have weak schema models. Users are allowed to add and removed data layers with ease and without constraint. The problem is that ensuring the integrity of such a model is very difficult. The schema layer is there to provide business rules and logic to ensure that he concrete data layer remains valid. We must also remember that the concrete data layer is there to help users do their work. Superfluous data items, unnecessary details and cumbersome graphic layers can impede users in getting their work done. A well designed data schema can transform unrelated data items into a work-centered application.

We call the lowest layer in the data model the types layer. By types we mean hard computer program objects, in our case .NET assemblies. ViSit Anywhere is designed to be an easily extensible application. We wanted to be able to add extension modules simply by dropping an assembly in the program directory and then allowing the new functionality to be discovered. Since new programs invariably need new schema and concrete data, it became apparent that we would have to close the circle and make the basic executable an element of the data model also.

The ViSit Site Model

The ViSit site model was developed over the past 15 years at Géotech as its primary spatial filter. The model uses parallel hierarchies of themes and sites. Theme are logical objects representing the business rules associated with some geographic asset. Sites are collections of related geographic information. Sites contain the geometric objects displayed in the GIS graphic view, while themes provide logic, as well as an extension point for hanging associated alphanumeric data and editing tools.

The diagram to the left shows how the hierarchies are arranged. Each site is associated with a geometric boundary that contains the site's geographic information as well as all the geographic information contained in the site's children. Thus, spatial filtering is a simply tree pruning operation (in the site hierarchy), because when a parent site's boundary is not contained in the area of interest, the child sites will not be contained either.

Each site is associated with a single theme. The themes are also grouped into a hierarchy. The theme hierarchy is arranged such that information that is applicable to a larger scale appears higher in the hierarchy than those applicable to a lower scale. For example, we might have state and city information. The scale of the state would be larger than that of the city. The state theme would be associated with one or more state sites (containing for example, geographic information of state-level features). The city theme would be associated with city sites. The city sites are children of a state site. The theme provides scale-dependant spatial filtering. For example, when we are at a scale where state data is not to be displayed, all sites associated with the state theme can be ignored.

This simple example shows how parallel hierarchies of theme and site can be used to easily filter very large data sets.

The ViSit Anywhere Data Model

The ViSit site model provides much of the insight that underlies the ViSit Anywhere data model. ViSit Anywhere manages project data in three hierarchies as shown is the project explorer dialog (left) used during project configuration. The top level hierarchy is the project data, which contains a model of the assets that are being managed by the application. The second hierarchy is the schema data that describes the project data.

The ViSit site model is implemented by placing themes in the schema hierarchy and sites in the project data hierarchy. A special site linkage object in the project data hierarchy is used to make the connection between themes and sites.

Finally, there is a types hierarchy that describes the assemblies and classes that are used to implement theme and project data.

Typical ViSit Anywhere users might not notice the structure of the data model, as it is transparent to the user, however, it is important for application administrators to understand the basic principals, so that they can optimize the organization of the project. Developers looking to extend the application should also understand the data model, as the object model is a simple implementation of the data model.

An important thing to understand is that the data model was designed with change management in mind. ViSit Anywhere provides a change managed environment for multiple users performing concurrent editing out-of-the-box, that is, not additional modules or configuration are required. A robust data model means that all data items, types, schema object and asset model objects are all change managed. Thus when any change is applied to the project that change is automatically distributed to all users, simply by synchronizing with the project server. The change might be a change in types (by the addition of a new application), a change in the project structure by the administrator (a schema change), or a change in the actual project data (by an edit from a standard user) - changes at all levels are changed managed using the same basic infrastructure.

In a future post I will explain how the structure of the data model enables the implementation of the change management system and how change management allows ViSit Anywhere users to more easily maintain their important geographic data and to derive more benefit from them.

Monday, March 10, 2008

Introducing ViSit Anywhere

This article is also available in French.

This blog is all about bringing ViSit Anywhere, Géotech's new geo-engineering application, from a development project to full-fledged product that can be used to solve complex business problems.

If you are not already a Géotech customer, you might be asking who is Géotech, what is ViSit and how can this help.

First, Géotech is a small company in Garches, France that has been delivering Microstation-based geo-engineering solutions since 1989. We have delivered a wide range of special purpose engineering applications, but we have two principal products, ViSit and ViSit/Web. ViSit is a geo-engineering tool that can be configured for a wide range of applications including network management (water, natural gas, electrical, etc.) and cadastral management. In addition, ViSit is an extremely flexible product and it has been extended to a number of other application domains, such as tunnel construction and facilities managements.

Géotech's other major product, ViSit/Web. is built on top of ViSit. It is used to provide consultation and editing functionality to the users via their web browser. The combination of ViSit and ViSit/Web can provide a robust GIS platform for a applications with a wide range of user skill levels and needs.

ViSit (Visualization of Sites) is an application that executes inside of Bentley's Microstation CAD editor. It's really novel feature is its site model. The ViSit site model is a implementation of spatial indexing that is, at once, geometrical and logical. It is geometrical in that the site boundaries are used filter the available sites for any spatial operation. It is logical in that each ViSit project contains a hierarchical tree of theme-sites that allow both logical and scale based filtering. Using the ViSit site model, projects can display and maintain literally thousands of Microstation design files - even on low performance workstations. Contact Géotech for more information on the ViSit site model.

When Bentley released Microstation version 8, they changed the structure of their design files, which had a serious impact on much of the ViSit program logic. At that time, we decided that rather that churning through 10 years of legacy code, we would re-write ViSit. This decision meant that we could restructure the application to take into consideration all that we had learned from our many years of working with a wide variety of customers. It also allowed us to adopt the latest (and the greatest) technologies to provide a flexible, robust software system that could be tailored to the needs of our customers. The resulting product is ViSit Anywhere.

The first beta release of ViSit Anywhere scheduled is for the first quarter of 2008. In the next few weeks we will be rolling out the product to a select group of beta-testors. We will also be posting entries on this site that explains the new features of ViSit Anywhere, some use cases and some early feedback from the product.

As a prelude to the coming release, I would just like to quickly mention some of my favorite features that went into ViSit Anywhere.

  • It is implemented as a Microsoft .NET application. This has allowed us to integrate better with Windows and to provide the latest in user interface technologies, web connectivity and enterprise integration. The current release of ViSit Anywhere is based on the .NET framework version 3.0.
  • It uses a abstract CAD engine. Using Microstation as an editing tool provides users with all the technical drawing power they can ask for. Unfortunately, this comes at a cost, not only in terms of Microstation licenses, but also in terms of user training. In ViSit Anywhere, we still provide a tool for Microstation power users, but now we also provide a simplified environment for people who can contribute to the geo-engineering project, but who are not technical draftsman.
  • It provides disconnected editing. That is, users can download the data they need to do their job and head out on the road (or just sit at their desks). When the job is finished they can commit their changes to share them with their colleagues or they can just throw the project away. Multiple local projects provide an excellent environment for simulations, especially since all the required data is in the local project.
  • It provides simple, robust change management. Geo-engineering projects are often maintenance projects, where changes are (relatively) few and widely separated. This provides an excellent environment for optimistic change management. ViSit Anywhere provides user-based conflict resolution (as opposed to administrator-based systems) making change management very simple.
  • A customer friendly licensing scheme. Géotech provides a new licensing scheme, which should allow all users to have the ability to edit system data, without costing a fortune. In addition the new Géotech license server makes deploying the application extremely simple.
In summary, we think that ViSit Anywhere is a new breed of tool that will change the way companies think about GIS. This new workflow should allow them to extract maximum benefit from their geographic data.