Challenges and Opportunities of Big Data-Free-Samples for Students

Question: Analyzes the Challenges and Opportunities of big data. Answer: Introduction: Big data is referred as large capacity of information, and it includes both structured data and un-structured data that assist the business on daily basis. It must be noted that, amount of data is not important but what organization do with the data is actually matters[1]. Introduction of big data is considered as new revolution in the technology, but this technology impose various new challenges for both users and service providers. This paper analyzes the challenges and opportunities of big data. Lastly, paper is concluded with brief conclusion which state key points of the complete essay. About Big data: Big data is completely new term, and it includes the collection and storage of big quantity of information for evaluation. This technology is originates in starting years of 2000s when new definition of big data is provided by the industry analyst Doug Laney. Organizations collect the data from different sources which include transaction conducted by the business, information available on social media, and information available on machine-to-machine data. In the past, data storage is considered as big issue but now this problem is resolved by new technologies such as Hadoop[2]. Big data importance does not rotate around the fact that how much information organization has, but it mainly deals with the matter how organization deals with the data. Organization can collect the data from different sources and examine the data related to cost reductions, time reductions, etc.[3]. Organization is able to achieve following business related targets while combining the big data with high-powered analytics: Organization can determine the root causes of failures, issues, and defects in near real time. In other words, it becomes easy for organization to identify the threats on time and make changes accordingly, as it reduces the risk of uncertainty for organizations. Organizations can produce vouchers at the time of sale. Organizations can recalculate the entire risk portfolios in few minutes. It becomes easy for organizations to make calculations in context of risk portfolios in few minutes, before this technology organization spends days to make such calculations and it also affects the productivity of the organization. Detecting fraudulent behavior in the organization before it causes any harm[4]. It is important for organization to understands that primary value related to big data does not lies in the raw form of data, but primary value lies after the analysis and processing of the data, and also from the products and services that results from the analysis. Changes occurred in the big data technologies and management approach needs to be attended with the similar dramatic shifts in context of how data supports the decisions and products/services innovation[5]. Challenges and opportunities related to big data: It is easy to understand the opportunities related to the big data. However, this approach also reflects various challenges and because of these challenges, big data and technologies related to big data are not used by various organizations. As per one survey, 55% of big projects are completed by the organizations and second survey repeats these findings. This paper provides both challenges and opportunities related to big data[6]. Challenges related to big data: Following are some challenges related to big data: Hadoop cause hardship- Hadoop and other surrounded ecosystems of tools are used because of their ability to deal with the high volumes. It is not easy to use or manage the software. However, technology related to big data is completely new because of which it is not possible for data professionals to manage this technology in effective manner. It must be noted that Hadoop actually requires extensive internal resources for the purpose of maintenance, and there are number of organizations which contributes maximum resources for technology instead of solving the actual big data problem. As per one survey, almost 73% of the respondents stated that understanding this technique was the biggest challenge reflected by the projects of big data[7]. Scalability- while using the big data it is difficult to scale up and down on demand. There are number of organizations which are failed to consider how fastened big data project can show growth. Holding a project on constant basis for the purpose of adding additional resources cut down the time for data analysis. Workloads related to big data also tend to be bursty, and it makes the decision of resource allocation more difficult. Challenges related to big data changes as per the solution[8]. Lack of talent- organization feels that there is shortage of talent in the context of big data, which means not only the shortage of data scientists and analysts, who also possess proper knowledge and understanding for identifying the valuable insights. Generally, vendors of big data resolve this issue by providing their own educational resources. Actionable Insights- more data does not always results in actionable insights, and a key challenge for data handling teams is to identify a clear objective of business and also the proper sources of data to collect and evaluate for meeting the identified objectives. Once data team identified the key patterns for the same, then business must make preparations to act and ensure necessary changes for the purpose of gaining business value. Data Quality- quality of the data is not the new issue, but the actual issue is storage of original information. Dirty data charges huge to the organizations. There are number of common causes of dirty data which are addressed by the organizations. Additionally, big data techniques also help the organization in cleaning the data[9]. For the purpose of using big data analytics in most effective manner, it is necessary to have data that is accurate and complete. Data which is not reliable generally leads to wrong conclusion. Issue arises when errors related to users continuously hit the data sets. Security- another challenge for big data is the security of the vast data, and some of these specific challenges are stated below: There must be authentication of user for every team and member of the team who is able to access the data. Restricting the access of data on the basis of need of user. Must record the history of data access and also fulfill the other regulations. Use of proper encryption related to data. Opportunities related to big data: Big data is not a regular approach, but it is the most important approach for the organizations. As stated by Forbes, big data is turn into a movement, and now big data is more contributing in the decisions of the organization than before. Organizations need talented and skilled people for the large amount of data they hold. Some opportunities related to big data are stated below: Projections for big data: as stated by the IDC, market related to the big data will be value $46.34 billion by 2018. Technology related to big data and associated services market is assumed to grow at a compound annual growth rate (CAGR) of 23.1% from 2014 to 2019. It is estimated that annual spending of this market will reach $48.6 billion till the year of 2019. Hadoop is already considered as synonym of big data, and it is estimated that market related to Hadoop will grow at 58.2% CAGR between 2013 and 2020[10]. Estimations made by IDC are completely positive. It is also stated by Alex Rossino, a senior principal research analyst at Deltek that investments in cloud computing and big data technologies are increasing. Detleck forecasts that spending is increase by the federal agencies on services of cloud computing from $2.4 billion during fiscal 2015 to $6.2 billion by fiscal 2020. IDC also stated that the availability of talent for the purpose of analyzing the big data will actually determine the future market. Demand of big data in the organization increases the opportunities for people to develop their skills in this field and serve to the organization. Implementation of big data: as per one survey, clear evidence is present which reflect that this technique is moving out from its stage of experiment. There are number of organizations which are uses the big data for the purpose of production and almost 60% of them believes that big data is necessary for their business. It must be noted that, almost 25% of respondents are implementing the solutions proposed by big data in their organization. Jobs related to Big Data- as stated by Forbes, top five industries across the globe hiring people who possess big data skills. Below stated graph reflect the job openings in different industries Forbes further stated that the organizations demand related to sales representative that actually have skills to sell this technique. Big data professionals such as Information Security Analysts, Management Analysts, and Information Security Analyst are also in demand by the organizations. This table states the growth in demand in context of these job positions: Job Titles Growth in context of job demand Developers of software 94.86% Engineers of systems 93.19% System analysts 142.64% Information security analysts 279.69% Management analysts 162.92% Demand for big data skills- demand related to professionals is increasing because of the increase in investment. Organizations realize the value of data analytics and because of which they are looking for professionals who can deal with this data. Various job portals provide the clear indication of demand of big data jobs. Although this big data is a broad term and in job listing this term is not generally used by organizations[11]. Frequent changes occurred in technology, and big data is recently replaced with new technique that is real time. This does not mean that demand related to big data skills is now low. It just means that keywords have been replaced[12]. It is clear from the above facts that big data reflects various opportunities such as opportunity related to investment, jobs, etc. Conclusion: After considering the above facts, it can be said that big data is considered as large volume of data, and it includes both structured data and un-structured data that assist the business on a day-to-day basis. Big data reflects both opportunities and challenges for the organizations. Big data is the technology which mainly provides support to the organization in managing the large amount of data and this management includes data evaluation, data storage, data processing, etc. it also prevents data duplicity. References: SAS, Big Data, https://www.sas.com/en_us/insights/big-data/what-is-big-data.html, accessed on 27th February 2018 CRA, Challenges and Opportunities with Big Data, https://cra.org/ccc/wp-content/uploads/sites/2/2015/05/bigdatawhitepaper.pdf, accessed on 27th February 2018. James Manyika, Michael Chui, Brad Brown, Jacques Bughin, Richard Dobbs, Charles Roxburgh, and Angela Hung Byers. Big data: The next frontier for innovation, competition, and productivity. McKinsey Global Institute. May 2011. Computational Social Science. David Lazer, Alex Pentland, Lada Adamic, Sinan Aral, Albert-Lszl Barabsi, Devon Brewer,Nicholas Christakis, Noshir Contractor, James Fowler, Myron Gutmann, Tony Jebara, Gary King, Michael Macy, Deb Roy, and Marshall Van Alstyne. Science 6 February 2009: 323 (5915), 721-723. Materials Genome Initiative for Global Competitiveness. National Science and Technology Council. June 2011. Qubole, Big Data Challenges and Opportunity, https://www.qubole.com/resources/big-data-challenges/, accessed on 27th February 2018. Qubole, HADOOP IS HARD, https://www.qubole.com/blog/hadoop-is-hard/, accessed on 27th February 2018. Jonathon Buckley, (2015), Infographic: 5 Crucial Considerations for Big Data Adoption, https://www.qubole.com/blog/big-data-adoption/, accessed on 27th February 2018. Jonathan Buckley, (2015), Causes of Dirty Data, https://www.qubole.com/blog/causes-of-dirty-data-and-how-to-combat-them/, accessed on 27th February 2018. Acadgild, (2017), Big Data Job Opportunities In 2017 And The Coming Years, https://acadgild.com/blog/big-data-job-opportunities-in-2017-and-the-coming-years/, accessed on 27th February 2018. NAP, Opportunities and Challenges for Big Data and Analytics, https://www.nap.edu/read/23654/chapter/4, accessed on 27th February 2018. Bhadani, A., Jothimani, D. (2016), Big data: Challenges, opportunities and realities, In Singh, M.K., Kumar, D.G. (Eds.), Effective Big Data Management and Opportunities for Implementation (pp. 1-24), Pennsylvania, USA, IGI Global. SAS, Big Data, https://www.sas.com/en_us/insights/big-data/what-is-big-data.html, accessed on 27th February 2018. CRA, Challenges and Opportunities with Big Data, https://cra.org/ccc/wp-content/uploads/sites/2/2015/05/bigdatawhitepaper.pdf, accessed on 27th February 2018. James Manyika, Michael Chui, Brad Brown, Jacques Bughin, Richard Dobbs, Charles Roxburgh, and Angela Hung Byers. Big data: The next frontier for innovation, competition, and productivity. McKinsey Global Institute. May 2011. Computational Social Science. David Lazer, Alex Pentland, Lada Adamic, Sinan Aral, Albert-Lszl Barabsi, Devon Brewer,Nicholas Christakis, Noshir Contractor, James Fowler, Myron Gutmann, Tony Jebara, Gary King, Michael Macy, Deb Roy, and Marshall Van Alstyne. Science 6 February 2009: 323 (5915), 721-723. Materials Genome Initiative for Global Competitiveness. National Science and Technology Council. June 2011. Qubole, Big Data Challenges and Opportunity, https://www.qubole.com/resources/big-data-challenges/, accessed on 27th February 2018. Qubole, HADOOP IS HARD, https://www.qubole.com/blog/hadoop-is-hard/, accessed on 27th February 2018. Jonathon Buckley, (2015), Infographic: 5 Crucial Considerations for Big Data Adoption, https://www.qubole.com/blog/big-data-adoption/, accessed on 27th February 2018. Jonathan Buckley, (2015), Causes of Dirty Data, https://www.qubole.com/blog/causes-of-dirty-data-and-how-to-combat-them/, accessed on 27th February 2018. Acadgild, (2017), Big Data Job Opportunities In 2017 And The Coming Years, https://acadgild.com/blog/big-data-job-opportunities-in-2017-and-the-coming-years/, accessed on 27th February 2018. NAP, Opportunities and Challenges for Big Data and Analytics, https://www.nap.edu/read/23654/chapter/4, accessed on 27th February 2018. Bhadani, A., Jothimani, D. (2016), Big data: Challenges, opportunities and realities, In Singh, M.K., Kumar, D.G. (Eds.), Effective Big Data Management and Opportunities for Implementation (pp. 1-24), Pennsylvania, USA, IGI Global.