Creating a high-performing online database

Hi there,

I have an idea for an online database that services a large number of people. I've been studying it for a while and it seems feasible to me to create it and get people to populate it. It will need time to grow but eventually it will get there.

The model I'm looking at is IMDB, the depth of information is fascinating, yet it's fast, not so easy to use though, but it's pretty usable!

What do you think I need to create a database an online database like IMDB. I know that IMDB power comes from it's information, not the design of the site. This is something I kind of figured out.

But what I need to know is the best tools to publish database contents on the web, retrieve it in that fast way like IMDB. I'm sure that I will need to create data entry logs for my users to populate the database.

What programming languages you suggest? development environment? approaches?

your contribution is highly appreciated.

Regards,
Jalil

Comments

[Todd Hoff]

dabbledb.com might work. Google docs might be made to work. Or you can get an Amazon account and store data on S3. Google App Engine could be a usable choice when they lift their caps. Zoho now has a cloud SQL database (http://cloudsql.wiki.zoho.com/Zoho-CloudSQL-Architecture.html) that may be of interest.

[Ryan]

ok, but what kind of database do you want to create? And how are you going to get people to populate it? Anyway it's very important to find the right way. Because often we look so long at the closed door that we do not see the one which has been opened for us. So Good luck to your in your choice!

[Martin]

Take a look at www.recs4free.com
This will allow to design your own online database with unlimited users.

[bnewport]

We'd had good success with very high volume web sites using a combination of a conventional database with a IMDG in front of it. The IMDB is first preloaded with the hot records from the DB and then services all reads and lazy pulls what is not there from the DB (watch the #connections to the DB!). It also services all writes and commits those to replicated memory and then uses Write behind to buffer writes and then periodically flush 'dirty' records to the database in batch. The IMDG basically acts as the data service layer for the application in front of it. The application interacts with the data through the IMDG. The biggest site in production with us using this model does about 8k page views/sec peak right now and they know they can scale out as load grows to what ever. Response times for fetching and updating data dropped 10x after making the upgrade from their old model.

Billy Newport, Distinguished Engineer @ IBM
Chief Architect, IBM WebSphere eXtreme Scale

[juzcat]

Hi Billy
how does one go about creating a very high write intensive database that scales
We have manage to get our systems to 300+ TPS - financial systems but thereafter we are using ORACLE RAC to get availability. Adding more nodes in a RAC cluster despite what Oracle maketing says, seems to slow us down
eg using one 8 WAY machine with a SAN attached we can get 300 TPS
Going to 2 x 8 WAY in RAC we get 150 TPS
How do you get around Storage being the bottleneck
thanks

[atif.ghaffar]

juzcat, try another database.

[Andrew Shenderuk]

Hi!
I know many ways to create online database, but necessary to know php,sql...at present you don't have a time for this, therefore use http://mytaskhelper.com is the ease way to create dynamic website.
Steps to create online database:
1)Goto http://mytaskhelper.com
2)register
3)create new project
4)create new form
5)you can add fields to form(You need to know what type of data you will be presenting)
6)Then goto Widget Builder menu, you will see two windows, copy this code and paste to your site(blog, home page, etc.)
7)that's all)), enjoy

Hope it will help to everyone
Thanks
Andrew Shenderuk