[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: ozone & modularity
On Mon, 29 Jan 2001, Jean-Marc Vanel wrote:
> Ozone needs more documentation, especialy a high-level description of
> the architecture. UML documentation would be nice, like diagrams on the
> prowler site which are nice; but more is
> needed, not just diagrams generated from the code, need also sequence
> and/or collaboration diagrams.
Well, I perfectly agree. However, I'm not able to do all this by myself. And
I'm not sure if this would be a good idea anyway. I mean, for several years
just my ideas and just my code makes up ozone. I believe that it would be a
good think if other programmers would try to understand the design and the
code to find my mistakes and giving their fresh, new ideas. Making the
requested documentation could be a good starting point for this. Of course, I
would be more than happy to help with this by answering related questions on
the list. This would start a lot of discussions about all aspects of ozone
which would be another good thing.
>
> Ozone needs modularity. The broad functionalities should be modular and
> layered: basic object storage, XML storage, client-server aspect,
> transactions, XPath queries, etc.
If nothing else, ozone is truly modular and layered.
The contract of the store back-end, which handles persistency, clustering and
locks, is defined by the StoreManager and ObjectContainer interfaces. We have to
implementations of that (although one is not supported currently) which proves
that it is actually possible to plug-in new back-end stores.
The kernel just handles communication, transaction/thread management,
permissions and object invocation. Its entire interface is defined by the Db*
command objects, which represent a client->server request.
The client side layer does connection pooling and API related things (ODMG, XA,
etc.)
The XML storage works on top of this. It is just an application of ozone. No
changes to any ozone interface has been done in order to store XML in ozone!
All tools that work on top of the persistent DOM (XPath, XSLT) are not part of
ozone and are just used as they are (Xalan, Xt, etc.)
If you have a look at the source tree you will find that inside those layers
there are clearly separated modules.
>
> The limitations of the current implementation should be listed.
I agree. I will maintain this list. Just send your list of limitations and we
will discuss about.
>
> The are many good free software projects, but API's are lacking, which
> would enable us to take the best of each.
Maybe I lost you here. Do you mean "standard" APIs, like ODMG and JDO, that
would (eventually) allow to be vendor independent or do you mean APIs in
general?
>
> _____________
>
> For a quick solution to my problem of stroring big documents, I could
> use code similar to DOMStore() in XML/Client.java. I just have to use a
> ContentHandler that creates DOM objects and stores them right away in an
> Ozone transaction.
>
> But anyway it seems that the Sax storage strategy should be enhanced to
> avoid memory limitations. Is it easy to change the storage policy so
> that memory problem are
> avoided during load of large document? Parhaps by commiting from times
> to times?
> Based on available memory, a chunk size for partial transactions could
> be computed. Also to handle
> disk space limitations, an estimated storage size should be computed for
> an XML file, in order to forbid storage if too big.
>
>
> Ozone as it stands now can only store small XML files, about 1Mb, even
> lots of them. But with many files, you cannot do a global XPath request
> on all those documents without Prowler.
This is true. ozone/XML is currently not suited for XML files > 1MB. As stated
on the toDo list this needs to be reworked. I do have some ideas in this
regard others hopefully too, we should start discussing this effort.
>
> Basic concepts of Ozone and Prowler are nice, but the architecture
> either not enough modular or
> not enough documented.
see above
>
> Modularity is really important. For instance, what we need for the
> "Worldwide Botanical Knowledge Base" project is a way to make XPath
> queries with sufficent response times on a 15Mb document.
> Probably to achieve this, an indexing of textual content is needed.
see above
>
> With ozone, we have no big documents, no indexing of textual content.
> But we have transactions, client-server, Java objects persistence, ODMG
> API's.
> Since our requests will be done in a servlet, and updates will happen
> every few weeks or months, we don't need all this fonctionality, but it
> is here, and it probably adds complexity to the possible implementation
> of enhancements in the storage modules.
>
> And of course, modularity makes for nice architectures, reusable and
> better maintainable.
> So if I could get advice on how to use the basic storage +XML +XPath
> alone, I'm willing to work on these parts.
See above. ozone/XML is an application of ozone. It is build on top of ozone.
Do you get my point? ozone is an objectbase. It is possible to use it to store
XML. Today it doesn't do this in the most efficient way. I do have many ideas
how to improve this. But at the end this always will be an application of the
objectbase ozone. Of course it is possible to make a persistent DOM in other
ways, like IPSI PDOM (without transactions, client-server, proxies) for example,
but this has nothing to do with ozone then.
>
> After that, I could work on text indexation. Maybe for this it is
> possible to rely on Ozone infrastructure:
> create the words index as an extra XML document, this way:
> <index__><word1><id>objectID1</id>... etc
> and then translate internally an XPath request with contains() into
> another without, provided that we have internally a function
> objectID(node) .
>
> ______________
>
> Exploration of the code:
>
> org.ozoneDB.core.Env (environment of a ozone database server)
> instanciates an object of interface:
> org.ozoneDB.core.Store
> which is implemented by:
> org.ozoneDB.core.wizardStore.WizardStore
> which is implemented by:
> org.ozoneDB.core.wizardStore.ClusterStore
ClusterStore does not implement anything
> where the function:
> public DxBag recoverClusterIDs()
> looks in the storage directory for clusters
ClusterSTore is the only entity that actually deals with cluster files
> where the function:
> protected void storeData( Object obj, String key )
> actually stores a Cluster object
see above
Is something wrong with this?
Falko
>
> --
> <person>
> <firstName>Jean-Marc</firstName>
> <lastName>Vanel</LastName>
> <motto>Veni, vidi, convici</motto>
> <conference>9th International World Wide Web Conference - Amsterdam,
> May
> 15-19, 2000
> <a href="/ozone-users/01-2001/http://www.www9.org/">site</a>
> </conference>
> <project>Worlwide Botanical Knowledge Base -
> making botany available on Internet
> <a href="/ozone-users/01-2001/http://wwbota.free.fr/" >site</a>
> </project>
> <a href="/ozone-users/01-2001/http://jmvanel.free.fr/>home page</a>
> <a href="/ozone-users/01-2001/mailto:jmvanel@free.fr">mail (possibly put "wwbota" in
> subject to
> route your mail in relevant folder)</a>
> </person>
>
>
>
>
>
>
>
>
> --
> <person>
> <firstName>Jean-Marc</firstName>
> <lastName>Vanel</LastName>
> <motto>Veni, vidi, convici</motto>
> <conference>9th International World Wide Web Conference - Amsterdam,
> May 15-19, 2000
> <a href="/ozone-users/01-2001/http://www.www9.org/">site</a>
> </conference>
> <project>Worlwide Botanical Knowledge Base -
> making botany available on Internet
> <a href="/ozone-users/01-2001/http://wwbota.free.fr/" >site</a>
> </project>
> <a href="/ozone-users/01-2001/http://jmvanel.free.fr/>home page</a>
> <a href="/ozone-users/01-2001/mailto:jmvanel@free.fr">mail (possibly put "wwbota" in
> subject to route your mail in relevant folder)</a>
> </person>
--
______________________________________________________________________
Falko Braeutigam mailto:falko@smb-tec.com
SMB GmbH http://www.smb-tec.com