Annabelle McCulloch

Big data visualisation refers to the techniques and tools used to present the large and complex data into such a way that it’s easy to read and understand. Some examples include: Heat maps which is used to show the amount of data points or activities across different regions and categories.   Network diagrams helps to visualise relationships and interactions in data for example social connections/data flows etc.  Geospatial maps show the mass amount of geographical data with traditional data sets to provide spread apart analysis.  Stream graphs show trends and patterns across loads of different categories and regions.   Same for the parallel coordinates which also show patterns but also correlations across numerous different variable.  Chord diagrams help to identify clusters, patterns, trends etc to help with making intense decisions and also helps with analysis etc.  https://www.geeksforgeeks.org/what-is-big-data-visualization/#what-is-big-data-visualizat...

Data mining is the process of using statistical analysis and machine learning to reveal hidden patterns or odd things in large datasets because of this you can help make important decisions and predict what’s going to happen. it’s where you take data such as structured data, an image, video, text etc and train it, deploy and serve it then you can get actionable insights and application events.  The techniques are classification which is used to organise data into different classes or categories so it trains a model on labelled data and uses it to predict the class. Regression is used to predict numbers or continuous values based on relationships so it finds the function or model that best fits the data to make accurate predictions.  Clustering is used to group similar data uncover patterns or structures in the data without any classes or labels.  There is also association rule, anomaly detention, time series analysis, neural networks, decision trees, ensemble methods and ...

 One of the greatest challenges is storage with insane amounts of data generated everyday. Also because Unstructured data can’t be stored in traditional databases.  Processing big data is also a problem which refers to reading, analysing etc or useful information from raw information because of this the changing from all this data to finding all the useful parts is very challenging.  Security is another problem because non-encrypted info is more likely to be stolen or damaged which makes it such a large concern for organisations.   https://www.simplilearn.com/challenges-of-big-data-article#:~:text=Storage,be%20stored%20in%20traditional%20databases.

Making sure that biased data is not used such as making sure it’s accurate and also up to date can help to limit the bias that is seen through but data especially for the healthcare and financial systems.  Large businesses could think about using more modern techniques and tools to help deal with the large mass of data sets, this gets rid of duplication of data. Also they could additionally use machine learning technology, improve its development and marketing strategies etc.  Changes in the Regulatory environment means that they should keep up to date with data privacy needs  so they’re not storing information they don’t need. This will save them money and time because they won’t have to pay fines for holding unnecessary data. Also because they are ahead of other businesses then they can use this leverage for competitive advantage.  https://kms-solutions.asia/blogs/top-challenges-related-to-big-data-and-how-to-overcome-them

Big data helps organisations and also policy makers to develop answers to society’s issues and also to help aid social change e.g monitoring climate changes such as natural disasters to social welfare programs and humanity aiding projects etc. An implication of it would be the cost of implementing big data solutions which makes it challenging for some businesses. Also because it can be biased it means that it can be discriminatory towards certain people of society which is primarily true in the healthcare and financial systems, biased data sets leads to biased results.  https://www.harvardonline.harvard.edu/blog/pros-cons-big-data#:~:text=From%20monitoring%20climate%20trends%20and,and%20drive%20positive%20social%20change.

Collecting and analysing massive velocity of data increases the risk of unauthorised access, data leaks or cyber attacks because of this it creates privacy and security risks for individuals. For individuals this means more risk and fear of technology. However Data privacy and technology explores the risks and helps to offer solutions and strategies to manage data responsibility.  https://www.harvardonline.harvard.edu/blog/pros-cons-big-data#:~:text=Collecting%20and%20analyzing%20large%20volumes,strategies%20for%20managing%20data%20responsibly.