Re: wish: could composite store info about datastores that have been changed?

Marja wrote:
> Axel Hecht wrote:
>> Marja wrote:
>>
>>> I wrote code that updates datastore changes to Web too. I'm using 
>>> several composite stores for different purposes. One composite is for 
>>> reading (because that you can do from the Web directly) and some 
>>> others for writing and updating. However, as far as I understand the 
>>> composite does in no way support finding out which stores included in 
>>> the composite need refreshing e.g. which stores in the composite 
>>> changed when I asserted something.
>>
>>
>> Asserting into a composite DS is considered harmful. I strongly 
>> suggest that you directly put your arcs into the datasource they 
>> should go to.
> 
> I realized that after some experimenting, but why then offer that 
> interface? Or at least document this warning anywhere with the interface?

All our interfaces are deprecated without replacement. They're not as 
good as they should be, and the separation between reading and writing 
is one of the things we have on our queue. Tentatively, this is not 
really decided upon yet.

> And actually the documentation for which store is written to in a 
> composite is wrong anyways: I think the documentation says write to last 
> one but it actually writes to the first one or vice versa. But that does 
> not really matter because there was no way to control that something was 
> last or first if the store was already in the list.
> 
> I still try to use composites, but now several of them for different 
> reading, writing, and updating purposes as I thought they would offer 
> some benefits, e.g. be able to have store assertions etc. to a default 
> store if the others did not work and keep track of things so I can just 
> give one object to observers. But the composite does not give much help 
> for anything.
>>
>>> Would it be possible to keep track and tell which one of stores in 
>>> the composite need refreshing after the composite RDF was modified by 
>>> doing assertions, modifications etc. to the composite? This would 
>>> help to optimize some Web traffic.
>>
>>
>> This optimization should be handled by the datasource itself. Just 
>> make it store a dirty flag. There is no need to do this in the 
>> composite DS.
> 
> I could not find a dirty flag interface for datasource:
> http://xulplanet.com/references/xpcomref/comps/c_rdfdatasource1namexmldatasource.html 
> 
> 
> I guess you mean I keep track of changed datasources myself and hope I'm 
> in right context after all the observations when I finally need to 
> refresh to have the updated info. (I think the context actually works in 
> and out when systematically using openDialog parameters for it but it 
> would be so nice to do it in a simpler way.)

One way to do that is to just create a js wrapper impl on the 
in-memory-datasource and work on that. And that could hold a dirty bit 
and do the required serialization.

Unless you like the idea below.

>>> I understand that when I use http to upload something to Web it may 
>>> be difficult to find that out but if I know which stores were written 
>>> locally with RDF store APIs I can keep track of the Web modifications 
>>> and save some unnecessary Web reading and uploading.
>>>
>>> Naturally it would also help if composites and other datastores would 
>>> do the uploading in the Web without my code. But that is a bit more 
>>> work. Has someone done this already as a package?
>>
>>
>> How do you upload?
> 
> I have tried different ways for HTTP both read and write. Now I use 
> nsIUploadChannel and stream for upload, unfortunately my operator does 
> not accept HTTP so testing has been not so easy but I did manage to do 
> some in another site. I think adding ftp should be easy now too but 
> haven't done it yet.
> 
> I also use XMLHttpRequest for GET to read stuff from Web to temporary 
> local files so that I can use remote datasource RDF API for updating. I 
> also wrote XMLHttpRequest code for POST and PUT but haven't tested that 
> thoroughly only parts. I'm not sure anymore if authentication was done 
> automatically with XMLHttpRequest or if I should do something similar 
> I'm doing with uploadChannel and nsIAuthPrompt but coudn't figure out how.
> 
> I did try nsIDownload and WebBrowserPersist too for downloading but 
> there were some problems in writing the loaded stuff to a local file so 
> I finally gave up on it.
> 
> I'll do some more testing (maybe need my own http server) and then hope 
> to be able to put the code to CVS (and then try to optimize it better).

The length of your story and all the "I need to test this on another 
server" is exactly the reason why there is no code in Mozilla that does 
that. It just wouldn't work reliably, or just on a very confined set of 
servers.

IMHO, it would be much more promising to actually use a server-side RDF 
engine for writable remote RDF from the start instead of trying to hack 
around http. Not that I have any cycles to do that either.

> Marja
> (some day hoping to get back on focusing on semantic Web and social 
> bookmark/annotation user interfaces and not on protocols :-) )
> 

Yeah, that'd be nice.

Axel