Cool Links: Links of projects worth watching
Virtualization Infrastructure
A future replacement for vmware could be Xen which eventually should become faster than vmware; although currently it still has a way to go according to this paper. It even has its own network mgmt console much like the paid for vmware console. It can display all the remote machines and their performance. Fortuanately vmware is not going to quit without a good fight. Currently vmware player and vmware server are free; not to mention that they have vmware infrastructure 3 shipping which is FAST!. Not to be out done, Solaris 10 is now offering zones and brandz. Now that Amazon’s Elastic Compute Cloud (Amazon EC2) is based on Xen and the with the virtual extensions (Intel VT and AMD Pacifica), we can run Windows without OS support as required with vmware. Xensource has a very interesting demo of their management console which looks like it could be exactly what we require to manage the epics core.
Was tracking down how well linux scales and ran across this anouncement from Sun that vmware is being ported to solaris 10. Granted this linux/solaris 10 comparison paper is a year old now but one can see how much more performance our XP guests would run under solaris 10.
Hardware Technologies
We are still tracking the hardware extensions that Intel and AMD have released. Intel has already shipped their newest chips with VT and pacifica extensions are reported to be shipping very soon. These hardware extensions should allow us to get additional performance out of future servers we deploy.
I picked up a Linksys wireless-G Broadband Router WRT54GS at Sams earlier in the year as it was a linux platform. It is about the cheapest ($50) method of getting an embedded plaform that one can control having gone the soekris route last year. If you are looking for some additional performance (increased range for wireless, vlans, ipsec, etc) and in general more control over your cable or dsl connection, then this little guy is for you. Some of them have USB ports so this might be a good candidate for a forensic harvesting engine with a portal USB 2.0 drive. Here are some screen shots of one of the available firmware upgrades.
Steve forwared me this hosting company that is doing grid level hosting starting at $89/month. The infrastructure appears to be blue arc storage servers with the cluster running NFS. Here is a library of papers describing their combined SAN/NAS product offerings. A couple of interesting points that struck with me is that Ben Rockwood purchased an account and found the disk storage to be slower than a thumper. Unless they don’t have 48 spindles then this is a real shocker or the thumper is that good. The other thing about Blue Arc was they are gearing up for grid/virtualization wars with their blades. I read the product sheets and once again, marketing is making claims of performance that make no sense unless you are running Infiniband and I have my doubts those work without some type of clustered overlay given the interupt/cpu bottlenecks of the pc platform.
Filesystem Wars!
ZFS or a clustered file system? Don’t know what ZFS is yet? Then watch these two movies to see ZFS self healing features or perhaps this on the basic usage of what ZFS is. To scale ZFS one must use ISCSI to take advantage of really large systems. Then there is lustre – a beast of a file system! In a recent testing they claim – “testing on production supercomputing clusters, Lustre achieved an aggregate parallel I/O throughput of 22 GB/s utilizing in excess of 90% of the available raw I/O bandwidth.” Finally not to be outdone, s3 is all the rage (simple torage service / simple scalable storage). Amazon has introduced their version of it which might be a good solution for off site storage for Redfile. Dr dobbs had an interesting comparison of perfromance between amazon s3 and scp (SSH). BTW, have you ever wished that you had made a filesystem larger or perhaps just a bit smaller then GParted is the tool for you. This live cd can handle most linux and windows filesystems without data corruption.
Backup Software
Looking for simple backup systems based on tar instead of using dump but not full blown like amanda. Bacula might do the trick to write to tape at native speeds. Rsync is hard to beat for getting the data to the backup device although NFS and cp -a did a decent job last night at over 120GB/hr through the NFS partition from epics-core01 (28) to epics-core04(superpc). That is smokin! Our DS3 will do about 5MB/s with ssh so the bottleneck is going to be WAN bandwidth andnot our software.
Syncing up windows and unix
Here is the problem – We have windows computers and unix computers and want to keep an active backup or perhaps just keep our directories in sync. There are loads of tools but rsync and unison are time tested candiates and many others are based on these two. Rsync works by taking your files and breaking them into blocks and creating md5 sums and then only copying the blocks that are different according to the md5 hash. It works over ssh as the transport and is fast. Unison on the other hand works similar to rsync but offers the ability to keep both trees at the same revision level. In other words, you work on one tree and the other tree but now want the merged changes to exist on both trees. How cool is that!
Put the Fun Back into Computing
That is the tagline of Distribution Watch but it is appropos in this case. There is a distribution here for anyone including a livecd for security folks, a linux XP clone, or perhaps a distribution to write a distibution. And not to be outdone, the knoppix based live-cds are well represented here which include things like helix a forensics tool kit. If you are eager to create your own distribution, linux live cd/usb is a great place to start.
Security for the rest of us.
How about a credit size appliance with two ethernet ports that protects windows XP from anywhere. For those security hackers that want some additional network tools, Whax and Auditor have mereged into Back Track. This looks interesting for the windows/linux users. Its an encrypted filesystem using AES-256.
Databases that Scream
Was looking into Sun Microsystems X4500 code named thumper when I came across two DB solutions offering ultra high throughput. Kognitio WX2 and greenplum. Greenplum is claiming scanning 1TB/minute. There are some that have their doubts on the Greenplum claim however.
Web and nothing but Web
Ajax is fantasic. How about some of these cool applications. Nothing like turning someones browser into a thick client with the click of a link.
VOIP Technologies
A friend called me today on skype and I didn’t know other than the 000 area code that something was up. I was on vonage and he was talking on skype… not long distance charges and clear as we were in the same room. Though some have concerns about skype because its a p2p networking technology.
S3 Storage from Amazon Web Services
http://www.amazon.com/gp/browse.html?node=16427261 The future of our storage infrastructure.
