Opportunities in handling big data are exploding and universities need to develop courses in cloud computing.

Cloud computing is a phrase that’s been very much in use since 2009 when the prospects of Web 2.0 loomed over the horizons. It is a concept whose history goes back to the 1960s. At present it is very much in use, and we can even envisage new areas of research and work that will develop in this area. Cloud computing is an extension of the concept of distributed computing – which is the process of running a program or application over many computers connected by a network. The Internet makes this process easily achievable even for the general user. However, this simple definition masks the complexity of the process and its possibilities.

Cloud futures

One important change we may encounter in the future is that computing could become a utility service. Just like we pay for electricity, we may have to pay for computing power. There are important differences: While electricity is a commodity produced at a centre and transported to the user, computing power is stored in the servers and users connect to it through their networks.

Data needs to be transferred from the user to the server and processed there. Electricity is a stateless resource, whereas the data and its stages of processing (states) need to be stored and backed up in the server. Also, since the data is being taken in by the server, its usage should come with a promise of privacy, security and accessibility.

The need for keeping data secure drives many fields. For example, legal data has to be kept in-house. In the same way there are restrictions that banking and medical data should not be allowed to cross the borders of the home countries. This would necessitate the use of private clouds. On the other hand, in education, public clouds can be used for which one would pay less.

The whole bouquet, some of it yet to develop — public and private clouds, computing as a utility service and big data analytics — portends exciting and eventful developments for the future — new career paths, new courses and new technology.

Studying cloud computing

There are a host of courses and workshops offered by various agencies on cloud computing. Are these actually touching the core areas? “The student should certainly look for the core concepts underlying cloud computing. These include concepts of virtualisation, map-reduce programming on Hadoop clusters, nosql database concepts and big data analytics,” says D. Janakiram, professor of the department of computer science, IIT Madras. “What we are talking about is programming in, say, 10,000 nodes,” he says. While universities in developed countries have already introduced courses in cloud computing, these are still not part of what is offered in Indian universities. “Introduction of new programmes in cloud computing… by universities requires many approvals and generally requires a great awareness of the technology both from educators and aspiring students. This awareness is slowly building up and this could lead to introduction of these programmes by universities… however, universities are offering electives in this subject,” says Dr. Janakiram.

IIT Madras has a course in cloud computing. The course is aimed at providing the required conceptual understanding of cloud computing and big data processing. It looks at cloud computing as a logical extension to the concepts of distributed and grid computing, service-oriented computing. Concepts of virtualisation, large-scale elastic computing on clusters which include map-reduce programming on Hadoop clusters, underlying concepts of Hadoop file system, concepts of stream processing, storing of big data in nosql databases, concepts of big data analytics, and concepts of service-oriented computing are taught. Dr. Janakiram, who has taught the course twice, says, “The assignments involve the use of cluster manager software such as Zookeeper for coordinating distributed processes, stream processing using Storm abstraction. The students use the Hadoop cluster donated by Yahoo, which resulted in the set up of Yahoo Grid lab.”

The future of the field is promising but a lot also needs to be done. It is predicted that new fields, such as Data Science, will come up with the advent of cloud computing and big data. It is important to equip students with the required skills in cloud computing and big data analytics to be able to fully exploit these opportunities. A new breed of specialists called data scientists are likely to emerge in the near future.

Jobs are already getting created in the fields of data structures, big data analytics, Hadoop and map-reduce programming. The future promises many more applications, for example in the areas of banking and telemedicine.