Directory view mode storage suggestions (scratching my head, any insights/observations might prove useful)...

D

Dimiter_Popoff

Guest
I am close(r...) to finishing the new dps browser, at least as its
function as a file browser. I have been working on it on and off for
a few years now, between other tasks of higher priority.
Now I have reached a point where I have to decide how to store
each directory viewing details, those which are unique per directory,
like column widths, view mode (details with hieroglyphs (icons),
thumbnails etc).
To put them in the \"global store\" (one can save and retrieve various
record types there, an app can initialize the active/default/factory
records it needs in only a few lines of code with plenty of available
variations) does not appeal to me, that will be a record per directory
and would take maintaining coherency with the directories etc. I don\'t
like it. Has anyone seen it done like that? (e.g. MS in their
\"registry\" though I doubt they do it there?)
The option I think I\'ll go for is to have a file in each directory
carrying this info. It is not a lot, 1k will be plenty I guess..
I have a . and a .. file carrying the path as text and the path
above since the dawn of dps (back in 1994 I think, may be 95).
These have never been used, they only get created when creating a
directory. Dps keeps the directory files for the entire path
of a directory which is open open so there is no need to refer to
these. What if I squeeze that info into the . file? I\'ll preserve
its present contents, just past it. It will take up no disk space,
a typical cluster is a few kilobytes so the . file won\'t allocate
anything in addition to what it already has.
Anyone seen that done this way? Or similar? Any thoughts on why
not to do it this way?
I am resigned to the fact that this will mean inability to remember
the directory view mode on non writable media.

I know this is lengthy and not quite on topic but everyone here
uses that sort of thing and there is quite a diversity of experiences,
I guess I have made up my mind (in a great part while writing the above)
but I am still in head scratching mode and someone saying something
might prove important to what I do next with this.

======================================================
Dimiter Popoff, TGI http://www.tgi-sci.com
======================================================
http://www.flickr.com/photos/didi_tgi/

Dimiter
 
On 9/16/2022 3:51, Don Y wrote:
On 9/15/2022 5:42 PM, Dimiter_Popoff wrote:
....

Thanks Don, that multiuser reminder was key.

Note that you can also implement a \"default settings\" (unless overridden)
in that way.  And, impose a given set of defaults on all folders, even
those that have previously been customized -- by simply \"forgetting\"
all of the cached entries (forcing the \"default\" to be used)

That I already have in place, each object can have its
active/default/factory parameters in the global store under a path(s)
of its own choice. Parameters being like signed long, unsigned dual,
list of objects etc., each carrying also unit information (no unit
if not applicable). For example, the browser object takes from
there the log file directory and the log file name (don\'t remember
exactly why it has two entries for that), which are stored as
\"list of objects\", each object being a \"string\".
Or the visualization object (e.g. dir_view_details) takes from
there the hieroglyphs per each file (each a list_of_objects,
like @rectangle, @line, @bitmap etc.) , under
...../common_browser/active/file_types/bitmap/gif ,jpg etc.,
except active there is also a default and a factory path; upon
doing \"gstinit\" (the highest level) a sequence is processed etc.
Then the entire \"global store\" is a file, heavily cached but
a file which can be backed up etc., it being a \"disk\" image
itself with very space saving parameters (e.g. block size
is only 64 bytes, cluster size is 1 block I think etc.).

Obviously I got carried away but that\'s on my mind at the moment :).
 
On 9/15/2022 10:13 PM, Dimiter_Popoff wrote:
On 9/16/2022 3:51, Don Y wrote:

Note that you can also implement a \"default settings\" (unless overridden)
in that way.  And, impose a given set of defaults on all folders, even
those that have previously been customized -- by simply \"forgetting\"
all of the cached entries (forcing the \"default\" to be used)

That I already have in place, each object can have its
active/default/factory parameters in the global store under a path(s)
of its own choice.

OK. But my point was that any folder whose settings have slipped out of
the cache would have to be initialized with the defaults.

*So*, if you want to \"reset\" all folder settings, you just purge the cache;
no need to walk the file system changing values in \".\" nodes!

Parameters being like signed long, unsigned dual,
list of objects etc., each carrying also unit information (no unit
if not applicable). For example, the browser object takes from
there the log file directory and the log file name (don\'t remember
exactly why it has two entries for that), which are stored as
\"list of objects\", each object being a \"string\".
Or the visualization object (e.g. dir_view_details) takes from
there the hieroglyphs per each file (each a list_of_objects,
like @rectangle, @line, @bitmap etc.) , under
..../common_browser/active/file_types/bitmap/gif ,jpg etc.,
except active there is also a default and a factory path; upon
doing \"gstinit\" (the highest level) a sequence is processed etc.
Then the entire \"global store\" is a file, heavily cached but
a file which can be backed up etc., it being a \"disk\" image
itself with very space saving parameters (e.g. block size
is only 64 bytes, cluster size is 1 block I think etc.).

I don\'t expose a filesystem to my applications. Instead, they
make queries of a relational database using schemas that they
have defined -- or, that other apps have defined as a \"convention\".
(these are published in their interface documentation)

The thinking, there, was that I don\'t want to have to require apps
to parse raw files (what if they encounter something that isn\'t
consistent with their expectations? e.g., \"February 31\" as a date?).

Instead, I *guarantee* to each client that the data it receives
(retrieves) will satisfy the constraints placed on those data
when they were accepted by the RDBMS (e.g., Feb 31 violates
the \"date\" constraint). Further, that the RDBMS will ensure
that data stored remains intact.

This lets apps cut out lots of data validation steps when they
consume this sort of \"input\" -- the validation has been done *in*
the RDBMS. It also means the apps don\'t have to worry about
performing that validation *correctly* (if three different
consumers of a datum impose different validation constraints on
it, how can you be sure *what* the \"right\" data should be?)

It also puts an *active* entity in the process. E.g., if
\"something\" changes a parameter of interest to a particular
app, then the RDBMS can notify that client of the fact that
a change has occurred -- and the client can re-query the RDBMS
for a more current value (using whatever criteria the *client*
considers to be appropriate)

> Obviously I got carried away but that\'s on my mind at the moment :).

I\'ve been hashing out a similar set of problems for my namespace
implementation; there, getting O(1) performance is a desirable goal.
 
a a <manta103g@gmail.com> wrote in news:24ffa3fc-5152-4949-9bb2-
471e7cae7979n@googlegroups.com:

if you mean knowledge representation engine

say the above once again in clear wording

You are a meaningless twerp.
 
On Friday, 16 September 2022 at 18:03:01 UTC+2, DecadentLinux...@decadence.org wrote:
#fuckoffyoustupiddog
 
On 9/15/2022 5:51 PM, Don Y wrote:
On 9/15/2022 5:42 PM, Dimiter_Popoff wrote:
Hmmm. The cache approach sounds better indeed. Because of the multiuser
angle.

That\'s a win -- but I thought the idea that folks likely don\'t care (remember?)
how they viewed the 114th folder prior!  Esp as the contents may have changed
in that time (e.g., I often have to make more space for LONG filenames if
I am doing something that creates them, but wouldn\'t want to have to rely
on them always being lengthy)

I do have an \"object_log\" object already which logs accesses to
directories and has various means for searches into it (e.g when you
start typing matches get offered), a variation of it might be the way.

Hash the full pathname to the folder and use that as a key to see if
a suitable entry exists in the cache.

Note that you\'ll have to purge the cache of any folders for which entries
exist that may later be deleted! And, if you support the renaming of
directories, either \"fixup\" those entries (if present) in the cache
or purge them.

(and all entries BELOW the affected one)
 
On Friday, 16 September 2022 at 00:01:10 UTC+2, Dimiter Popoff wrote:
I am close(r...) to finishing the new dps browser, at least as its
function as a file browser. I have been working on it on and off for
a few years now, between other tasks of higher priority.
Now I have reached a point where I have to decide how to store
each directory viewing details, those which are unique per directory,
like column widths, view mode (details with hieroglyphs (icons),
thumbnails etc).
To put them in the \"global store\" (one can save and retrieve various
record types there, an app can initialize the active/default/factory
records it needs in only a few lines of code with plenty of available
variations) does not appeal to me, that will be a record per directory
and would take maintaining coherency with the directories etc. I don\'t
like it. Has anyone seen it done like that? (e.g. MS in their
\"registry\" though I doubt they do it there?)
The option I think I\'ll go for is to have a file in each directory
carrying this info. It is not a lot, 1k will be plenty I guess..
I have a . and a .. file carrying the path as text and the path
above since the dawn of dps (back in 1994 I think, may be 95).
These have never been used, they only get created when creating a
directory. Dps keeps the directory files for the entire path
of a directory which is open open so there is no need to refer to
these. What if I squeeze that info into the . file? I\'ll preserve
its present contents, just past it. It will take up no disk space,
a typical cluster is a few kilobytes so the . file won\'t allocate
anything in addition to what it already has.
Anyone seen that done this way? Or similar? Any thoughts on why
not to do it this way?
I am resigned to the fact that this will mean inability to remember
the directory view mode on non writable media.

I know this is lengthy and not quite on topic but everyone here
uses that sort of thing and there is quite a diversity of experiences,
I guess I have made up my mind (in a great part while writing the above)
but I am still in head scratching mode and someone saying something
might prove important to what I do next with this.

======================================================
Dimiter Popoff, TGI http://www.tgi-sci.com
======================================================
http://www.flickr.com/photos/didi_tgi/

Dimiter
if you mean knowledge representation engine

say the above once again in clear wording
 
On 9/15/2022 3:01 PM, Dimiter_Popoff wrote:
Now I have reached a point where I have to decide how to store
each directory viewing details, those which are unique per directory,
like column widths, view mode (details with hieroglyphs (icons),
thumbnails etc).
To put them in the \"global store\" (one can save and retrieve various
record types there, an app can initialize the active/default/factory
records it needs in only a few lines of code with plenty of available
variations) does not appeal to me, that will be a record per directory
and would take maintaining coherency with the directories etc. I don\'t
like it. Has anyone seen it done like that? (e.g. MS in their
\"registry\" though I doubt they do it there?)

Questions of scale: how many directories/folders are likely to exist
in a given namespace? And, *realistically*, how many is the user
likely to CARE about?

E.g., I have a look at Windows\\System32\\drivers\\etc a few times each
year. But, I\'ve NEVER looked inside Windows\\System32\\drivers\\en-US!
I can probably say that for the many (most?) of the folders on a
given machine.

My point being, how likely is the user to care about the \"presentation\"
of a folder he viewed 6 months ago? Or, more importantly, a folder
he viewed 113 *folders* ago?! I.e., it may be just as effective to
maintain a cache of the N most recent \"settings\" on the assumption
that a smaller number of folders are \"of interest\" to a user
than the entire set possible. If you\'ve viewed 113 folders already,
would you mind having to re-set the parameters for the 114th one
that you viewed *prior*?

The advantage of this approach is the resources required are fixed,
regardless of the size/complexity of your namespace -- it\'s ALWAYS
113 sets of parameters. You can choose to store these in a fixed
size entry in your \"global store\"... or, in a \"hidden file\" (of
fixed size) on your medium.

The option I think I\'ll go for is to have a file in each directory
carrying this info. It is not a lot, 1k will be plenty I guess..
I have a . and a .. file carrying the path as text and the path
above since the dawn of dps (back in 1994 I think, may be 95).
These have never been used, they only get created when creating a
directory. Dps keeps the directory files for the entire path
of a directory which is open open so there is no need to refer to
these. What if I squeeze that info into the . file? I\'ll preserve
its present contents, just past it. It will take up no disk space,
a typical cluster is a few kilobytes so the . file won\'t allocate
anything in addition to what it already has.

The downside is you risk dicking with parts of the *real* filesystem
structure (what happens if there\'s a glitch while doing so? The
cache approach MIGHT cause the contents of the cached data to become
invalid but the structure of the filestore isn\'t at risk)

Anyone seen that done this way? Or similar? Any thoughts on why
not to do it this way?
I am resigned to the fact that this will mean inability to remember
the directory view mode on non writable media.

The cache approach is impervious to media type. It could even be
volatile -- at some inconvenience to the user.

Do you have to support any notion of \"users\"? I.e., Bob might want
things displayed one way while Fred wants them displayed another?

The cache approach scales linearly with increased users; the filestore
approach means your filestore has to reflect the number of users
(conceptually a Bad idea)
 
On 9/16/2022 3:24, Don Y wrote:
On 9/15/2022 3:01 PM, Dimiter_Popoff wrote:
Now I have reached a point where I have to decide how to store
each directory viewing details, those which are unique per directory,
like column widths, view mode (details with hieroglyphs (icons),
thumbnails etc).
To put them in the \"global store\" (one can save and retrieve various
record types there, an app can initialize the active/default/factory
records it needs in only a few lines of code with plenty of available
variations) does not appeal to me, that will be a record per directory
and would take maintaining coherency with the directories etc. I don\'t
like it. Has anyone seen it done like that? (e.g. MS in their
\"registry\" though I doubt they do it there?)

Questions of scale:  how many directories/folders are likely to exist
in a given namespace?  And, *realistically*, how many is the user
likely to CARE about?

E.g., I have a look at Windows\\System32\\drivers\\etc a few times each
year.  But, I\'ve NEVER looked inside Windows\\System32\\drivers\\en-US!
I can probably say that for the many (most?) of the folders on a
given machine.

My point being, how likely is the user to care about the \"presentation\"
of a folder he viewed 6 months ago?  Or, more importantly, a folder
he viewed 113 *folders* ago?!  I.e., it may be just as effective to
maintain a cache of the N most recent \"settings\" on the assumption
that a smaller number of folders are \"of interest\" to a user
than the entire set possible.  If you\'ve viewed 113 folders already,
would you mind having to re-set the parameters for the 114th one
that you viewed *prior*?

The advantage of this approach is the resources required are fixed,
regardless of the size/complexity of your namespace -- it\'s ALWAYS
113 sets of parameters.  You can choose to store these in a fixed
size entry in your \"global store\"... or, in a \"hidden file\" (of
fixed size) on your medium.

The option I think I\'ll go for is to have a file in each directory
carrying this info. It is not a lot, 1k will be plenty I guess..
I have a . and a .. file carrying the path as text and the path
above since the dawn of dps (back in 1994 I think, may be 95).
These have never been used, they only get created when creating a
directory. Dps keeps the directory files for the entire path
of a directory which is open open so there is no need to refer to
these. What if I squeeze that info into the . file? I\'ll preserve
its present contents, just past it. It will take up no disk space,
a typical cluster is a few kilobytes so the . file won\'t allocate
anything in addition to what it already has.

The downside is you risk dicking with parts of the *real* filesystem
structure (what happens if there\'s a glitch while doing so?  The
cache approach MIGHT cause the contents of the cached data to become
invalid but the structure of the filestore isn\'t at risk)

Anyone seen that done this way? Or similar? Any thoughts on why
not to do it this way?
I am resigned to the fact that this will mean inability to remember
the directory view mode on non writable media.

The cache approach is impervious to media type.  It could even be
volatile -- at some inconvenience to the user.

Do you have to support any notion of \"users\"?  I.e., Bob might want
things displayed one way while Fred wants them displayed another?

The cache approach scales linearly with increased users; the filestore
approach means your filestore has to reflect the number of users
(conceptually a Bad idea)

Hmmm. The cache approach sounds better indeed. Because of the multiuser
angle. I do have an \"object_log\" object already which logs accesses to
directories and has various means for searches into it (e.g when you
start typing matches get offered), a variation of it might be the way.
Log files being on a per user basis etc. I could even have each access
logged together with the presentation data and thus be done with it.
Thanks Don, that multiuser reminder was key.
 
On 9/15/2022 5:42 PM, Dimiter_Popoff wrote:
Hmmm. The cache approach sounds better indeed. Because of the multiuser
angle.

That\'s a win -- but I thought the idea that folks likely don\'t care (remember?)
how they viewed the 114th folder prior! Esp as the contents may have changed
in that time (e.g., I often have to make more space for LONG filenames if
I am doing something that creates them, but wouldn\'t want to have to rely
on them always being lengthy)

I do have an \"object_log\" object already which logs accesses to
directories and has various means for searches into it (e.g when you
start typing matches get offered), a variation of it might be the way.

Hash the full pathname to the folder and use that as a key to see if
a suitable entry exists in the cache.

Log files being on a per user basis etc. I could even have each access
logged together with the presentation data and thus be done with it.

You could also support a physical token (e.g. SD card) that users use
to personalize the machine to their settings -- for THEIR experiment(s).

[I realize you may not have the hardware in place for that on current
offerings but could do something similar with a mountable share
(per user, per experiment, etc.) OnMount() would just import any
objects that it encounters, there (with gatekeeping logic to ensure
nothing malevolent can get in) without having to make changes to
the base instrument\'s store]

> Thanks Don, that multiuser reminder was key.

Note that you can also implement a \"default settings\" (unless overridden)
in that way. And, impose a given set of defaults on all folders, even
those that have previously been customized -- by simply \"forgetting\"
all of the cached entries (forcing the \"default\" to be used)
 

Welcome to EDABoard.com

Sponsor

Back
Top