Now its Microsofts turn

Few days ago, I posted about all the criticism Yahoo is facing in Yahoo is here to stay.. or is it?

Now this time, its microsoft's turn. Everyone seems to be disappointed with the giants progress so far. See the articles on CNET and and by Henry Blodget.

Echoing the similar sentiments as I expressed earlier for Yahoo!, I feel it would be too bad for search industry if Microsoft were to perish here. Those with more grey hair seem to have similar opinion [1] [2].

A solution that always works for incompetent leaders

Dont know the real problem? Know the real problem, but not bold enough to attack it?

As a leader what do you do in such a situation? Well, you cook up lot of non-issues, solve the non-existent problems, and amuse yourself with the idea of having contributed towards a better tomorrow.

The case in point is the recent decision by IIT Bombay authorities to limit the usage of internet and intranet services. Please read the email message at the bottom of this post to get the context.

The authorities find that the student attendance in classes are not as high.. Academic performance is going down.

At the same time they observe that students are spending lot of time on internet.

You see a change, you see an effect. Bang! You label the change as cause of the effect.

Internet is just a medium. Its nothing more than a technology shift. It isnt for the first time that we are seeing a technology change. When newspapers came in, they ate into the time we used to spend on other things. When radio came, samething happened. Then television came. Today internet is here.

As engineers, it is incumbent upon us to embrace the new technology with open arms. We are the creators of these technologies.. if we dont believe in change, why did we come into engineering to begin with?

Take any industry.. First we used to commute by road.. which limited the distances we could travel. Then came the railroads.. And the air travel has made it possible to collaborate and connect with people spread all over the world.

With addition of new technologies, we subsume the possibilities of yesterday.

Now that it is clear that we need to embrace the new technology, the next question is how to make the best use of internet. This medium gives us great power. And together with it comes the great responsibility.

What do you do when you are endowed with a great responsibility?

You rise to the occassion and deliever.

What is being done in IIT is, no.. we cant handle so much responsibility.. Lets reduce the responsibility to the level we know how to handle.

Is this right message for your students? How do you expect we will continue to believe that you can impart in us the qualities that makes a great leader?

I am not saying leaders should not do anything, and let things take their own turn. No.. I agree that a technology shift demands an adaptation. And this adaptation would require several initiativies from the leaders of our institute.

What I am arguing here is that the approach being taken is a very short-sighted one. The right solution requires bold leadership, a visionary who can lead us in this time of change. May be someone with qualities in JFK, or Rajiv Gandhi..

It is a pity that in a country with a population of 1 Billion, we cannot find the right leaders for the nation's best institution. Helplessly, we have left it in the hands of a bunch of incompetent managers ready to compromise the future of next generation of promising citizens with temporary fixes.


Now, I dont have the maturity to lay out a grand plan that can be enacted upon with 100% success rate. Nonetheless, I know a few high level things that needs to be done. You need to teach people, help them imagine the possiblities for creating useful tools using the new technology. You adapt your course structure to the new medium. You show them the vastness of knowledge, far larger than Main Library, that they can now learn to shape up their future. You teach them how to broadcast themselves into the world, as it gets flattened and democratized with the widespread adoption of internet.

On the other hand, it is pretty obvious that the approach being taken is not the right one. If there is a new technology, and we dont know how to make best use of it, lets not allow our students to use it. By restricting its usage, we will be able to minimize its bad effect.

I wonder what happens when students graduate from IITs, and get unlimited access to the internet? Well, who cares? They are not our liability any more.

What did you say? We didnt teach our students how to handle great responsibility?

Give us a break. It was never a part of the IIT's curriculum.


From: General Secretary Academic Affairs < gsecaa@iitb.ac.in >
Date: Jan 19, 2007 6:32 AM
Subject: Minutes of the meeting on Network Usage by IIT Students
To: General Secretary Academic Affairs < gsecaa@iitb.ac.in>

Hi all,
Attached are the minutes of the meeting on Network Usage by IIT
Students. Please go through the mail. In case you have a suggestion
or concern, use the forum iitb.general to share it with students and
faculty, instead of personal mails.

For the unaware, iitb.general is a forum that is accessible to all
students, faculty and alumni and is used for discussions and exchange of
ideas. To access iitb.general , visit http://varta.iitb.ac.in and log in
with your netmon ID and password.


Minutes:
----------------------------------------------------------------
INDIAN INSTITUTE OF TECHNOLOGY, BOMBAY
OFFICE OF DEAN STUDENT AFFAIRS

19th January 2007

A meeting of the committee consisting of the following members were
held to discuss matters relating to computer and network usage by
students when the following members were present:

Prof. P. Gopalan, Dean, SA Convener
Prof. Anurag Mehra, Head, Computer Centre Member
Prof. Nand Kishore, Chairman, HWC Member
Prof. G.Sivakumar, Head, CSE Member
Prof. Raghav Varma, Warden, Hostel 10 Member
Prof. C. Amarnath, Ex-Dean, SA Member
Gen.Secretary, Hostel Affairs Member
Gen.Secretary, Academic Affairs Member
Mr. Swapnil S. Sachdev Invitee (MLC)
Mr. Parijat Garg Invitee

Item No 1:
Video streaming :
The committee decided that video streaming in inter and intra hostels
should be disallowed. For using this facility prior permission from
Head, Computer Centre should be obtained.

Item No.2:
Download limit:
The committee put a download limit of 3GB per month.

Item No.3:
Time based ban on internet access in hostels:
The committee had the requested to the students to give their inputs
in September 2006, but no inputs were received at that time, however
during the meeting the students proposed that internet access be
blocked between 2 am and 6 am. However, the committee decided that LAN
access only be available in hostels between 12.30p.m to 11 p.m.
Student dissent was noted. This policy would be effective from 26th
January 2007. It was agreed to review the policy after a period of two
months.

Item No.4:
Illegal Content :
The Head, Computer Centre brought to the notice of the students about
signing of an IT policy of the Institute by every student, which
says that they would use the IIT network to download any illegal
content on any computer connected to the Institute network for which
they are responsible. It was also discussed that the Institute
network is being used to disseminate "illegal materials" like films,
songs etc. by persons running ftp servers on their computers. It was
decided that it will be the task of the elected representatives
(councils) to report this to the Wardens for further action. It was
decided that a poster would be put up by 31st January 2007 after the
same has been vetted by Prof. Sivakumar and Head, CC. The poster
would explicitly spell out the consequences in case of violation
relating to the use of IIT network for the disseminating illegal
contents in any form.

(P.Gopalan )
Dean,SA

----------------------------------------------------------------

A big fraud.. or was I too lucky?

During some debugging, I was checking the access log on my web server.

Accidently, I noticed following (and many other similar) entries. Why would google, yahoo and microsoft all get interested in refreshing their index of my photo album at exactly the same time? Perhaps, some crawler out there is faking its identity.

I tried nslookup for these ips, and the info seem to correspond to microsoft, yahoo and google respectively. My network admin tried to convince me that I was lucky that all 3 search giants are simultaneously interested in me.

Likelihood of such an event is so small.. Assuming the refresh cycle of 30 days, and the scheduling epoch of one hour, the probability that all 3 companies would have scheduled a particular page in the same hour is something like (30*24)^(-3) = 3 out of a billion.

My probability estimation is missing some terms. You need another division by 20 Billion to account for "a particular page", and a multiplication with 30*24 for "the same hour".

I dont believe in so much of coincidence. Either there is a big fraud going on.. or the big giants have some collaborative projects going on..

65.55.209.52 - - [22/Jan/2007:19:15:25 -0600] "GET /~gkabra2/publish/summer2005/navahoPass/slides/P1040659.html HTTP/1.0" 200 14002 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"

74.6.86.107 - - [22/Jan/2007:19:29:34 -0600] "GET /~gkabra2/publish/summer2005/helen/slides/IMG_1050.html HTTP/1.0" 200 13570 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"

66.249.66.171 - - [22/Jan/2007:19:31:27 -0600] "GET /~gkabra2/publish/summer2005/4thJuly/slides/P1050115.html HTTP/1.1" 304 - "-" "Mozilla/5.0 (compatible; Googlebot/2.1;+http://www.google.com/bot.html)"

Vertical search engine market

Search engine watch has interesting article on the rise of the vertical search engine market. This is a very good news for my company, Cazoodle, for we are building tools that will facilitate development of verticals.

However, this article seem to confine the scope of verticals to the services similar to Yahoo search bulider and Google custom search engine. These services allow you to narrow down your search to a pre-specified set of sites. Some of them also expose a few enhanced search functionalities like keywords to include/exclude, term weighting scheme, etc.

I would imagine verticals include many more search services.. ranging from people search to google finance.

It is interesting though that, inspite of the narrow definition, verticals have been projected to be 1B industry by 2009. I wonder what would be the total market size if you consider the broader definition of verticals.

On the other hand, this article seems to indicate that the job portal sites are losing their market share to other niche, more sophisticated job domain portals based on social networking. However, the giants like monster, careerbuilder seem to be taking counter measures.. forming alliances to stretch out their reach. I guess the problem is not in their design but in their reach to larger audience. Yes, thats exactly what our company is building. So stay tuned!

Yahoo is here to stay.. or is it?

Yahoo is facing a really bad time.. with its revenues going down quarter by quarter.. wall street is losing faith. But yahoo is such a giant.. I think Yahoo is here to stay.. or is it?

Read the wired article on how yahoo blew it. There are various interesting points to note.

First, which is not entirely a news, but Yahoo has been conservative in various acquisitions.. But I dont see it as incompetency of its execs. No one can justify the price tags for these hefty acquisitions.. YouTube going out for 1.65 B. Its valuation is far smaller.. Paying myspace 1 B just for serving ads.. Google current revenue through myspace is perhaps 10% of this price tag.

But you still have to do it.. As it turns out, this industry happens to be the one where winner takes it all. To emerge as winner, you have to consolidate, and kill competition. That requires very bold moves, and confidence of investors in your crazy decisions.

Second, acquiring two pieces of code doesnt imply you can now have both. Yahoo acquired inktomi and overture.. But it took a long time before yahoo could integrate them both.

Third, systems are often overhyped for their value.. This was true in overture's case.. While it was the best ad serving system, it was not ready for the global scale yahoo wanted to deploy it on. It required various re-engineering before yahoo actually started making money out of it.

The current problem yahoo is facing is because it doesnt have an equivalent of adSense. Google makes 20 cents per search.. Thats a lot of money. Yahoo has been developing Panama for a long time.. but its still not ready. I really wish it works, and produces great results. Because, even for a "winner takes it all" industry, it is important that there are more than one player.

Something that strikes me most is.. today people are so comfortable and talking so freely about advertising as source of revenue.. In general, I would imagine people to be critical of search companies for showing advertisements, and making money out of it. I find the comfort of costumers and analysts with this business model very surprising.

Consulting career

When graduating from IIT, I was mostly interested in business consulting jobs.. So much that I didnt even want to interview with any software company. I couldnt imagine myself writing programs.

However, better senses prevailed.. I came here to grad school. This is where I really started loving writing softwares. Techies rule the world.. Consultants seemed like the spectators betting on who will win the game, and developers seemed like kicking the field goal. I started losing respect for management career, and at this point.. I cant imagine myself doing anything other than writing softwares.

Given this background, you would perhaps be able to understand why I thoroughly enjoyed reading this blog post by Joel Spolsky. While he is trying to make an entirely different point, the scenarios he uses amuses me. :)

CBS 15 second

CBS has announced a 15 second video contest.. Here is my message to the world, on my 15 second video on You Tube:

data center on cargo containers

Data centers tend to be a huge deal these days.. The challenge comes specially because companies are now dealing with humongous amounts of data. Google, Yahoo, MSN, with the goals of caching every byte of digital media, and being able to search through them in fraction of second, require storage and processing of petabytes of data.

Not just the search companies, but any data intensive industry, such as financial trading firms, telecom industry, ecommerce sites like ebay, amazon, etc. need huge data centers.

The increasing demands have so far been coped with ad-hoc construction of data centers by putting together racks of cheap or high-end machines, depending on criticality of application. Dissipating heat and huge power requirements have been the fundamental issues in design.

Well, obviously, these ad-hoc solutions are still expensive and often far from optimal. Mobility is another special challenge. To start a data center in Paris, Google will have to send out its crew of engineers for days, often a cost companies cant bear.

So now industry has come up with a very indeginous solution. Cargo Containers. Yes, data centers that can be customized, and shipped in cargo containers.

Read this research paper from Microsoft:

http://www-db.cs.wisc.edu/cidr/cidr2007/papers/P35.pdf

Look at this promo video from Sun's Black Box project:

http://sun.feedroom.com/?skin=oneclip&fr_story=FEEDROOM162914&rf=bm

Wonder who else in industry is going to manufacture these containers.

[Update] I recently learned from this post from John Battelle (the author of famous "search" book) that even Google itself had similar project.. But it was abandoned because managers didnt puruse it far enough.

dual boot - gentoo on a winxp machine

I recently installed gentoo on a HP machine running win xp. The gui installer that comes with LiveCD is actually pretty easy to use. You first need to burn the iso image of LiveCD:

url

Now the problem is the disk paritioning part of the GTK installer. If you try the recommended partitioning, it fails. The reason being my xp is already using up 2 primary paritions. This leaves only 2 more paritions as MBR allows a total of 4 partition. You cannot create the extended paritions using this gui.

The workaround is to use fdisk/cfdisk to create the extended partition. (1) Create one (ext2) partition of ~100MB for boot. (2) Create an extended paritition. (3) Create a logical paritition for swap (change type using t) of about 2*ram-size. (4) Create a logical paritition (of ext2) for the rest of space. You will specify the sizes using the starting sector and +SizeMB.

Further, you will need to set up the filesystem on these parititions. Otherwise, the GTK installer does not recognize the paritition type created from fdisk.

Next, you should not forget to mention the mount points in GTK gui.

Another reason why your installation might crash is because of inappropriate selection of additional packages to install. Many of the packages need to be downloaded from internet. Some of which will fail to download. There are still others that require you to accept licences, etc. So be careful in selecting the packages to install.

The major problem was with GTK not allowing to create extended parition I guess. Rest of it were minor issues.

My laptop seems to be already using all 4 entries. I wonder how I would set up dual boot there.