The Software engineer: github.com/amukherjee
The Database engineer: github.com/amukherjee/shinobi
The Lazy Bum who hates doing repetetive crap and write lucratives python scripts to make his life a whole lot easier. ^_^
My current experiments involve making this managing 1.5 Tb of data. The most ive ever handled before. Perfomance Tuning these databases are a real challenge.
Currently our systems head is working on the hardware aspect. Trying to get higher IOps with fusion IO cards.
1.5 TB of data is hard to replicate and do backups on, specially if its growing at a contstant rate. Our application stack is geared towards analytics, so batch processing with hadoop is out of the question. I'm researching generating realtime computations at near lightning speeds as soon as i finish tackleing this nasty replication bit. ^_^
Life is too short i want to be on top. So much to research.... i wish i could make copies of my self to run in parallel.
I have been following 10gen's Mongodb. I am not confident enough to use mongo on production. So for now im just using for loggin purposes. I just wrote a monitering system in python and i didnt want to use beefy mysql to handle it so the monog document style flat file works fairly well.
I was following Akiban's higherarchal indexing, but havnt been able to commit myself to their projects whole heartedly. Since I became a Database Engineer, my free time is spent either researching new technologies, writing python tools to make my job easier, dealing with replication fuckups form mysql's federated tables being on different versions, dealing with temp tables ( I hate developers some times -_- ) , and oh yea... most importantly .... getting some much needed SLEEP.