Re: [eclipse-clp-users] Is referenced_record/2 related to accessing asserted clauses?

Hi Kish, Thorsten,

While using a store is much faster than using dynamic clauses when we only need to access a single item, the performance seems similar when we need to access every single item in the store / asserted database.
My code needs to do the latter, i.e. it needs to visit every item whenever it is accessing the database, so replacing the asserted database with a store would not seem to provide a lot of speed-up.

It seems that in order to tap into the store's superior performance, I would have to use the store in a way such that the key of the store is not merely an edge ID, which doesn't allow the code to constrain the search
through the items in the store, but rather a key that reflects some constraints that would narrow down the search to a subset of all the items in the store.

Is there an alternative which would not involve constraining the search and therefore would still access every single item in the store / database, while still providing a considerable speed-up?

Thanks,
Edwin

From: edcsgt_at_hotmail.com
To: eclipse-clp-users_at_lists.sourceforge.net
Subject: RE: [eclipse-clp-users] Is referenced_record/2 related to accessing asserted clauses?
Date: Mon, 26 Nov 2012 11:23:46 +0800





Thanks Kish and Thorsten for your help. I will try implementing your suggestions.

Regards,
Edwin

> Date: Fri, 23 Nov 2012 18:36:18 +0000
> From: kisshen_at_cisco.com
> To: eclipse-clp-users_at_lists.sourceforge.net
> Subject: Re: [eclipse-clp-users] Is referenced_record/2 related to accessing asserted clauses?
> 
> Hi Edwin,
> 
> I think the main point here is -- as Thorsten said -- dynamic clauses is 
> not the best way in ECLiPSe to do what you want.
> 
> On 23/11/2012 08:31, Edwin Wirawan wrote:
> 
> > I tried profiling a run of my code and the result showed that the majority of the time (~80%) is spent in referenced_record/2.
> >
> > My code asserts a lot of dynamic clauses at runtime and refers to these dynamic clauses regularly.
> >
> > Does referenced_record/2 spend most of its runtime accessing asserted clauses, or doing something else?
> 
> I assume you are running ECLiPSe 6.0 (or newer). The way dynamic 
> predicates are implemented was changed in that version, and it is now 
> implemented using the recorded database (record/2 and friends), and 
> referenced_record/2 is likely used to access the asserted clauses as you 
> suspect.
> 
> >
> > FYI, each asserted clause is of the form: edge(edgeN,
> > ...more content...), where edgeN is unique for each asserted edge clause (e.g., edge1, edge2,edge3,…).
> >
> > ~200K such clauses are asserted during a typical run.
> >
> 
> It looks like your clauses are basically data, and as Thorsten said, it 
> is not efficient to uss dynamic clauses for this. The user manual has a 
> chapter on non-logical storage that discuss the various ways you can 
> store and access data non-logically.
> 
> A slight correction to Thorsten's comments -- dynamic clauses are 
> "indexed" to some extent, i.e. there is some filtering of clauses that 
> have the same name and arity, because recorded/2, which is used to 
> access the clause, does do some filtering, but as Thorsten shows, it is 
> not that effective when you have a large number of clauses.
> 
> 
> > A slightly related question: I also called statistics/1. There were
> > 200K+ dictionary_entries, dict_hash_usage showed that the dictionary was
> >   full (8192/8192) and the dict_has_collisions had a value of
> > (196801/8192).
> >
> >
> > My guess is that the majority of 200K+ dictionary entries correspond to
> >   our asserted 'edge' clauses, which leads me to wonder if the large
> > number of dictionary collisions means that the dictionary is using the
> > functor 'edge' or some other conflicted
> >   value as a key instead of my unique edge ID values. Does that seem
> > right? Would the dictionary hash function have any difficulty
> > distinguishing IDs that share a common substring such as 'edge', and
> > thus cause the seemingly high number of collisions?
> 
> I am guessing that your 'edge' id are implemented as atoms (as opposed 
> to strings), as I don't think strings goes into the dictionary (and are 
> not indexed, so using strings as unique ids is a bad idea).
> 
> I suspect if you have so many different ids, you may be better off using 
> integers (I assume you generate your 'edge' ids by combining an integer 
> with the atom edge anyway), as integers do not need to be added to the 
> dictionary, and I suspect they might index better than the atoms as well.
> 
> >
> > I am also wondering whether there is a way to inspect the structure and
> > content of the dictionary and the database of asserted clauses, and also
> >   whether there's a memory profiler for ECLiPSe.
> 
> I don't think there is anyway for a user to examine the dictionary. For 
> asserted clauses, you can using listing/1 to list the clauses, but this 
> would not give you any information on how well the filtering works.
> 
> Cheers,
> 
> Kish
> 
> 
> ------------------------------------------------------------------------------
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> _______________________________________________
> ECLiPSe-CLP-Users mailing list
> ECLiPSe-CLP-Users_at_lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/eclipse-clp-users