Genral Web Comments
Wednesday, October 29, 2003
Venti: a new approach to archival storage
This paper describes a network storage system, called Venti, intended for archival data. In this system, a unique hash of a block's contents acts as the block identifier for read and write operations. This approach enforces a write-once policy, preventing accidental or malicious destruction of data. In addition, duplicate copies of a block can be coalesced, reducing the consumption of storage and simplifying the implementation of clients. Venti is a building block for constructing a variety of storage applications such as logical backup, physical backup, and snapshot file systems.
Coda File System: "What is Coda?
Coda is an advanced networked filesystem. It has been developed at CMU since 1987 by the systems group of M. Satyanarayanan. in the SCS department.
Why is Coda promising and potentially very important?
Coda is a distributed filesystem with its origin in AFS2. It has many features that are very desirable for network filesystems. Currently, Coda has several features not found elsewhere.
disconnected operation for mobile computing
is freely available under a liberal license
high performance through client side persistent caching
server replication
security model for authentication, encryption and access control
continued operation during partial network failures in server network
network bandwith adaptation
good scalability
well defined semantics of sharing, even in the presence of network failures "
InterMezzo File System Home: "What is InterMezzo?
InterMezzo is a new distributed file system with a focus on high availability. InterMezzo will be suitable for replication of servers, mobile computing, managing system software on large clusters, and for maintenance of high availability clusters.
For example, InterMezzo offers disconnected operation and automatic recovery from network outages. InterMezzo is an Open Source (GPL) project. InterMezzo entered the Linux kernel at version 2.4.15, but the latest code is always here.
On these web pages you can find the software, documentation, and other information you need to use InterMezzo or to help with its development.
What is Intersync?
Intersync is a new client server system that we have started to put together, it periodically polls the server and integrates any changes made into the client file system. Intersync needs more work before it can be used in production. Please assist us with this work. "
Distributed Data Storage on a LAN?
AFS (Score:3, Informative)
by Reeses (5069) on Wednesday October 29, @04:18PM (#7341374)
It's called the Andrew File System.
http://www.psc.edu/general/filesys/afs/afs.html
There's another alternative with a different name, but I forget what it's called.
[ Reply to This ]
Starting Score: 1 point
Moderation +2
100% Informative
Extra 'Informative' Modifier 0
Total Score: 3
Re:AFS (Score:1)
by Reeses (5069) on Wednesday October 29, @04:24PM (#7341443)
Whee.. replying to my own post... In addition to AFS...
Coda:
http://coda.cs.cmu.edu/
and InterMezzo:
http://www.inter-mezzo.org/
and there's a review here:
http://www.linuxplanet.com/linuxplanet/reports/436 1/1/
Although, honestly, a 5 second search on google for "distributed filesystem" would have turned this up.
Ah, well.
[ Reply to This | Parent ]
Re:AFS (Score:1)
by wetshoe (683261) on Wednesday October 29, @04:25PM (#7341456)
I'd have to agree, AFS is a great solution. I actually thought of this about a year ago, and I told a co-worker about it. He told me it had already been implemented, and as it turns out, it was, it's AFS.
AFS is actually pretty cool. You can run a file server that uses all this disk space of all the client machines. It's a great idea now, especially since most new machines come with 40GB hard drives, and most people don't use anything more then 5GB.
AFS is a wonderful solution to not only this problem that the poster is talking about, but it can be used in so many other interesting ways.
[ Reply to This | Parent ]
Re:AFS (Score:5, Interesting)
by fireboy1919 (257783)
(http://rustyp.freeshell.org/ | Last Journal: Tuesday April 29, @09:22AM)
In my experience, it's one of those "it would be a wonderful thing if it worked."
It requires it's own partition for each mount of it; you can't just share disks you've already got.
Setup also takes hours, and it probably won't work the first time. Online documentation is incredibly outdated, which doesn't help matters at all. It also takes a hefty chunk of computer to run it, because it requires a lot of watchdog type programs to fix the frequent corruption that happens to it as you use it.
The servers time has to be matched exactly, so it's also best if you've got an NTP server running and clients on all the machines.
It's also about ten times slower than Samba (which you might use instead to share with Windows machines), and it chokes when you try to move/copy/delete large files.
I tried it for a month before it completely corrupted it's own partition and I switched back to NFS and Samba.
I can't wait for the day when these problems are but a memory and such a system works flawlessly.
[ Reply to This | Parent ]
Re:AFS by pHDNgell (Score:2) Wednesday October 29, @05:26PM
Re:AFS (damn) by pHDNgell (Score:2) Wednesday October 29, @05:28PM
Re:AFS by Umrick (Score:2) Wednesday October 29, @05:31PM
Re:AFS by ipjohnson (Score:1) Wednesday October 29, @06:10PM
Re:AFS by rivaldufus (Score:1) Wednesday October 29, @06:30PM
Re:AFS (Score:1)
by kaybi (261428) on Wednesday October 29, @04:26PM (#7341462)
OpenAFS
http://openafs.org/ [openafs.org]
[ Reply to This | Parent ]
Re:AFS (Score:3, Informative)
by Strange Ranger (454494) on Wednesday October 29, @05:24PM (#7341980)
from karmak.org
AFS is based on a distributed file system originally developed under a different name in the mid-1980's at the Information Technology Center of Carnegie-Mellon University (CMU). It was first publically described in a paper in 1985, and soon afterwords was renamed to the "Andrew File System" in honor of the patrons of CMU, Andrew Carnegie and Andrew Mellon. As interest in AFS grew, CMU spawned the Transarc Company to develop and market AFS. Once Transarc was formed and AFS became a product, the "Andrew" was dropped to indicate that AFS had gone beyond the Andrew research project and had become a supported, product quality filesystem. However, there were a number of existing cells that rooted their filesystem as /afs. At the time, changing the root of the filesystem was a non-trivial undertaking. So, to save the early AFS sites from having to rename their filesystem, AFS remained as the name and filesystem root. In the late 1990's Transarc was acquired by IBM, who subsequently re-released AFS under an open source license. This code became the foundation for OpenAFS, which is currently under active development.
It's still running and running well at CMU (AFAIK - as of late 90's). Every student gets an "Andrew" ID. Actually the very first networked computer I ever logged into (other than dialing a bbs) was a 'node' on Andrew, in 1988. Very very cool at the time, and still is.
[ Reply to This | Parent ]
Re:AFS (Score:1)
by RageEar (57236)
(http://rageear.com/)
I worked for a company that used this on all of our *NIX based servers. I never ran into too many problems as an end user and when I did they were easily fixed.
However, talking to our IT director, he said it was one of the biggest pains in the ass to administer. He was forced into using this system by the VP of Engineering, because said VP was an alum from CMU. The IT director wanted nothing more to switch over to an NFS/filer based solution.
Just my two cents.
[ Reply to This | Parent ]
nbdsrvr
for windows.....
NBD Does this (Score:5, Insightful)
by backtick (2376) * on Wednesday October 29, @04:12PM (#7341293)
(http://wwwdotslashdotdotorg)
http://nbd.sourceforge.net/
"Network Block Device (TCP version)
What is it: With this thing compiled into your kernel, Linux can use a remote server as one of its block devices. Every time the client computer wants to read /dev/nd0, it will send a request to the server via TCP, which will reply with the data requested. This can be used for stations with low disk space (or even diskless - if you boot from floppy) to borrow disk space from other computers. Unlike NFS, it is possible to put any file system on it. But (also unlike NFS), if someone has mounted NBD read/write, you must assure that no one else will have it mounted.
Limitations:It is impossible to use NBD as root file system, as an user-land program is required to start (but you could get away with initrd; I never tried that). (Patches to change this are welcome.) It also allows you to run read-only block-device in user-land (making server and client physically the same computer, communicating using loopback). Please notice that read-write nbd with client and server on the same machine is bad idea: expect deadlock within seconds (this may vary between kernel versions, maybe on one sunny day it will be even safe?). More generally, it is bad idea to create loop in 'rw mounts graph'. I.e., if machineA is using device from machineB readwrite, it is bad idea to use device on machineB from machineA.
Read-write nbd with client and server on some machine has rather fundamental problem: when system is short of memory, it tries to write back dirty page. So nbd client asks nbd server to write back data, but as nbd-server is userland process, it may require memory to fullfill the request. That way lies the deadlock.
Current state: It currently works. Network block device seems to be pretty stable. I originaly thought that it is impossible to swap over TCP. It turned out not to be true - swapping over TCP now works and seems to be deadlock-free.
If you want swapping to work, first make nbd working. (You'll have to mkswap on server; mkswap tries to fsync which will fail.) Now, you have version which mostly works. Ask me for kreclaimd if you see deadlocks.
Network block device has been included into standard (Linus') kernel tree in 2.1.101.
I've successfully ran raid5 and md over nbd. (Pretty recent version is required to do so, however.) "
Network Block Device
NBD Does this (Score:5, Insightful)
by backtick (2376) * on Wednesday October 29, @04:12PM (#7341293)
(http://wwwdotslashdotdotorg)
http://nbd.sourceforge.net/
"Network Block Device (TCP version)
What is it: With this thing compiled into your kernel, Linux can use a remote server as one of its block devices. Every time the client computer wants to read /dev/nd0, it will send a request to the server via TCP, which will reply with the data requested. This can be used for stations with low disk space (or even diskless - if you boot from floppy) to borrow disk space from other computers. Unlike NFS, it is possible to put any file system on it. But (also unlike NFS), if someone has mounted NBD read/write, you must assure that no one else will have it mounted.
Limitations:It is impossible to use NBD as root file system, as an user-land program is required to start (but you could get away with initrd; I never tried that). (Patches to change this are welcome.) It also allows you to run read-only block-device in user-land (making server and client physically the same computer, communicating using loopback). Please notice that read-write nbd with client and server on the same machine is bad idea: expect deadlock within seconds (this may vary between kernel versions, maybe on one sunny day it will be even safe?). More generally, it is bad idea to create loop in 'rw mounts graph'. I.e., if machineA is using device from machineB readwrite, it is bad idea to use device on machineB from machineA.
Read-write nbd with client and server on some machine has rather fundamental problem: when system is short of memory, it tries to write back dirty page. So nbd client asks nbd server to write back data, but as nbd-server is userland process, it may require memory to fullfill the request. That way lies the deadlock.
Current state: It currently works. Network block device seems to be pretty stable. I originaly thought that it is impossible to swap over TCP. It turned out not to be true - swapping over TCP now works and seems to be deadlock-free.
If you want swapping to work, first make nbd working. (You'll have to mkswap on server; mkswap tries to fsync which will fail.) Now, you have version which mostly works. Ask me for kreclaimd if you see deadlocks.
Network block device has been included into standard (Linus') kernel tree in 2.1.101.
I've successfully ran raid5 and md over nbd. (Pretty recent version is required to do so, however.) "
Saturday, October 18, 2003
Friday, October 17, 2003
Friday, October 10, 2003
Tuesday, October 07, 2003
Home - Entropy 0.5.7 Build 341
ENTROPY is an annoymous and encripted person to person network. no central server.
Slashdot | Fracturing P2P Networks
how p2p networks are breaking apeart...ie freenet's little break away group.
How to Kill Spam Without the State
tells me what i already know, the gov't cant make laws agenst spam
Computer Laboratory - Downloads
a virtual mechine that is free, open source, and higher proforming than all others. kick ass
Monday, October 06, 2003
This blogger thing is deffently a nice way to keep a good, decriptive set of bookmarks and a blog but still, it can be a pain in the butt sometimes...
a person to person file sharing network with no center and encripted. its a lot liek freenet execpt doesent use the damn java virtual mechine...
Home - Entropy 0.5.7 Build 340: "ENTROPY stands for Emerging Network To Reduce Orwellian Potency Yield and as such describes the main goal of the project.
ENTROPY is developed as a response to increasing censorship and surveillance in the internet. The program connects your computer to a network of machines which all run this software. The ENTROPY network is running parallel to the WWW and also other internet services like FTP, email, ICQ. etc.
For the user the ENTROPY network looks like a collection of WWW pages. The difference to the WWW however is that there are no accesses to central servers. And this is why there is no site operator who could log who downloaded what and when. Every computer taking part in the ENTROPY network (every node) is at the same time server, router for other nodes, caching proxy and client for the user: that is You.
After you gained some experience with the ENTROPY network, there are command line tools for you to insert whole directory trees into the network as a ENTROPY site. So ENTROPY does for you what a webspace provider does for you in the WWW - but without the storage and bandwidth costs and without any regulation or policy as to what kind of content you are allowed to publish. Everyone can contribute his own ENTROPY site for everybody else to browse through. The contents is stored in a distributed manner across all available and reachable nodes and no one can find out about who put up what contents into the network [1]. Even if your node is not actively running, your contents can be retrieved by others -- without knowing that it was actually you who published the files. Of course this is only true if you do not publish your name (or leave your name or other personal data in the files you publish)
Have fun,
Juergen"
Googlewhacking: The Search for The One True Googlewhack
a fun little site that has little oddities about google. weird things like "showing retults 1 of 1 out of 175,000" and such. is the beheamouf known as google getting out of the controle of its owners? heh, i cant wait for people to connect this to a certin goobertorial canditate and the end of the world......
Yahoo! News - Library Wants to Put Chips in Books
it would make liberians work so much easyer but then again...just look at the posiblities for abbuse.....
By RON HARRIS, Associated Press Writer
SAN FRANCISCO - A civil liberties watchdog group is expressing concern over the San Francisco Public Library's plans to track books by inserting computer chips into each tome.
Library officials approved a plan Thursday to install tiny radio frequency identification chips, known as RFIDs, into the roughly 2 million books, CDs and audiovisual materials patrons can borrow. The system still needs funding and wouldn't be ready until at least 2005.
The microchips send out electromagnetic wave to a device that converts them to digital data containing a host of information. In libraries, the system is primarily designed to locate books in branches and speed up the checkout process.
Library officials say the "passive" chips would be deactivated as materials are taken from the library, thus preventing any stealth tracking of books — and by extension, people — off premises.
But Lee Tien, a staff lawyer for the Electronic Frontier Foundation, is concerned that the chips may have information that would remain accessible and trackable, whether by ingenious hackers or law enforcement subpoena. That, he says, would be a threat to privacy rights.
"If there's a technology for temporary deactivation, then presumably there's a system for reactivating it," Tien said. "Does the person have the ability to know if the RFID is on or off?"
Some of the foundation's concerns are rooted in the provisions of the USA Patriot Act, which critics have assailed as giving government the authority to obtain the records and threatening the privacy and First Amendment rights of library and bookstore patrons.
San Francisco's city librarian, Susan Hildreth, says the devices will help streamline inventory and prevent loss. Tracking people is not the goal, she insisted.
"It will not allow us to track people to their home or any location," Hildreth said.
She pointed out that several other major libraries, including the Seattle public library system, are moving to the chips instead of bar codes.
"Industry trends show that it's going to replace the bar code very shortly," Hildreth said. "We're trying to prepare for the future."
Seattle's 24 libraries are installing RFID tracking systems, with the first to be ready next spring.
The city of Santa Clara is installing RFID tracking at its main library and the county is considering a similar move.
Still, it's the opportunity for unauthorized tracking that concerns Tien.
"The issue is other people, other institutions. What will they do if the RFID is insecure?" Tien said. "We're talking about the imbedding of location trafficking devices into the social fabric."
Hildreth said San Francisco library officials may hold a public forum to discuss the chips further.
meh, me sleepy. this day sucked, went to try and find lisa to ask her to homecomming/out but couldent find the place, bored, feel sick, blah, overall shitty day. need to plan the next lan party too. was gonna to it this weekend so shafer could come but it was just too short of notice. next weekend sounds nice but im not sure. hmm.... hmm, more random stuff on my message board from rick and the other lanninites.... heh, always fun to read. cant wait till i get a new message board up. but i really have to wait to do anything till i get stuff straightened out with who is hosting my site and what i can do with it. 56k issoooo slow, still waiting for this image rick sent to load. oh well, squid with a GB or two of cach seems to help. well, it helps with the frequent stuff but thats about it. oh well. what i wish i could do was get squid to preload links from pages in the background so its ready when i need it. there are a million web acceleratios out there that do just that but i want squid to do ti for me. and do fun stuff like keep the cached copy of slashdot upto date and other sites i frequent. but i doubt that something like that will happen soon. its probalby been done but there are other thigns that take priroity over that. oh well. meep. its instering seeing how different me and ohzie;s skills have become. he writes shells for windows and makes windows actualy half usable. i work with the fun stuff of *nix and dive off the deepend into linux... hmm, its 12:15 but blah, no sleep yet, read or surf, read or surf....oh what a choice. speaking of which i still need to find a nice text to speach engen so i can lissen to slashdot and other important info when there is nothign better to do. ie while driving or in class. blah, time to read now, night
Mon Album v 0.8.6 one big assed lan party..... i do hope boblan is like this big in the future. but i dont think my basment is quite big enough......
Sunday, October 05, 2003
stuff thats going on....
ill be switching message boards beforfe too long. if you run accross one you like lemme know. iv already decided on slashcode or clone for the phlophsye's message board. its desigen is perfect for such dissuision. but for our pourpoues ill probably go with a genral use message board for lan stuff. if you run accross one that you like lemme know so i can take a look. btw, this is what i put in another post that relates....[quote] this board doesent really matter any more, im ditching it for a better one, probably mysql/php/perl based. i sent my isp an altomatium telling them to either offer competivly priced servies or im switching to another server. at the moment it looks like it would be yahoo. iv checked out a few others, prohosting, being the other one that looked half decent. but at the moment yahoo seems like the best deal. there will be another topic regarding this...[/quote] im hoping that they will make a compeative offer so i can stay local, but who knows, its quite probbile that its not cost effective to do so. in which case i understand. but mabey ill get really lucky and they'll let me do a limmited form of colocation...
gpmidi.net
its my message board for the moment, probalby gonna ditch it soon. has gpmidi.net stuff, school stuff, tech stuff, lan party stuff, and genral topsics
Slashcode: Slash Open Source Project
its the slashdot code....if only it wasent a daemon it might be doable on my current server....then again it might be, oh well, with no sql it doesnet matter...but that altomation of mine shoudl fix things up...
