global filesystem namespaces
Here’s the seed idea for a global filesystem: Using a user-space filesystem implementation, organize available data by hash and metadata. For example:
% mount -t gsfs -o hash=sha1.tiger.sha512.md5,block=1k,tree=tiger.sha1,dublin=simple /global
% ls /global
dublin/ hash/
% cd /global/hash
% ls
sha1/ tiger/ sha512/ md5/
% cd /global/hash/sha1
% ls
00/ 01/ 02/ 03/ 04/ 05/ 06/ 07/ 08/ 09/ 0a/ 0b/ 0c/ 0d/ 0f/ 0e/ 0f/
…
f0/ f1/ f2/ f3/ f4/ f5/ f6/ f7/ f8/ f9/ fa/ fb/ fc/ fd/ fe/ ff/
% cd /global/hash/sha1/0ffed1778ae30aa477b10a89110fea6db0ae144/
% ls # this is a content node:
content blocks/ hash/ tree/ dublin/
% cd hash
% ls
sha1/ tiger/ sha512/ md5/
% stat -c ‘%N\n’ md5/hash
md5/hash' ->/global/hash/md5/7f9d50980771e33bf2e0f27ee0845d7a’
% ls tiger
hash@ tree@
% cd ../dublin
% ls
creator/ title/ source/
etc.
The ideas are: (1) break up large namespaces heirarchically, (2) populate namespaces on demand for large spaces, (3) provide a logical heirarchy in which metadata resides, (4) cross-link redundant subtrees.
The whole thing should be very doable using a DHT.