top of page
StreamAnalysis_2880x1620.jpg

Media Analysis V2

Unleashing Insights: How NLP Techniques Illuminate Media Data V2

In our first exploration of historical data news articles, we performed sentimental analysis to get the overall trend of the media portrayal of a topic. The next step that we wanted to explore is connecting a large language model that will be able to give us a more insightful view of the historic data and allow us to discover new properties.

Current Work Flow

Searching news data base for relevant news articles based on key terms generated from a prompt.

Perform sentimental analysis assigning every article and abstract a score saving the scores to a file.

Summate all of the sentimental scores and create a historic visualizations.

Find points of interest using peak and trough detection algorithms 

Run a large token LLM and derive countless different properties from the articles, and get interesting insights!

Points of Interest Detection

Identifying the points of interest for analysis was the first problem that we wanted to solve. For this I decided to focus on the peaks and troughs, or the breaking points in the sentiment data at which an even of interest may have occurred. After which the relevant articles can be processed by the LLM, although processing all of the articles in increments is also an option we are considering, it is not resource efficient at larger datasets of articles.

Of course it is important to recognize hat there are several different ways of identifying important events in the media. But in this case we are using all instances of derivatives of 0 based of a smooth function align to the data.

Although we did arrive at the conclusion that there has to be a better way of detecting peaks and troughs in the data, as smoother datasets that are less volatile did not have 0 derivatives, as could be seen with the childbirths. If you have any suggestions or recommendations on how to improve the detection of points of interests feel free to contact us.

v2 aligment.png
v1 aligment.png
download (2).png

Adding Large Language Modeles

We are currently working to implement Large Language Models (LLMs) into the analysis. Although, harnessing the power of sentimental analysis, topic modeling, and entity recognition, we gain valuable insights into the ever-evolving social landscape. However, when we combine these analytical techniques with a robust large language model, a whole new realm of understanding emerges. It is through this synergistic approach that we unlock the ability to identify crucial points and conduct profound analyses. We firmly believe that integrating the LLMs into our methodology represents the next evolutionary leap in comprehending vast volumes of data, enabling us to extract meaningful insights and make impactful observations. LLMs are able to interpret the meaning and a more nuanced approach to pieces of text.

We believe this is a stepping stone in connecting artificial intelligence to the current flow of events. The amount of data being generated about events is far too great for any one person to even fathom, and the artificial gene has the ability to help us make decisions by helping us understand what is happening holistically. And even getting suggestions based on a holistic understanding of current events.

Embodied-AI_2880x1620_Lede-scaled.jpg
original-9a68c0df5680aad37bf30740a8fd0331 (1).jpg

Development Plans

We are planning on making an open-source project to publish social analysis reports on requested topics, as well as topics current important topics that will hopefully be able to give people an understanding of the current and historical state of media to understand and potentially understand the manipulations of media. In addition, we are currently developing a commercial platform for private use with more extended features, that would be unsustainable to support free publishing.

Datasets

If you are interested in the datasets that we used in this demonstration, they are free to download and use in your own research. If you have any additional inquiries, please feel free to contact me at nkudrin@omni-group.dev.

Networks_2880x1620.jpg
bottom of page