Since October 2015, we have been involved as scientific partners in two FFG-funded projects (KIRAS security research program). The first project has developed a tool called Foresight-Strategy Cockpit. This software should enable the Austrian ministries to improve crisis management strategies by becoming aware of unexpected trends and future scenarios in security issues. The second project has integrated WebLyzard, a software tool for an automated analysis of online news and social media sources (comments on articles, Facebook postings and Twitter statements), to analyze the media representation of pressing societal issues and citizen’s security perceptions.The WebLyzard software was used in a case study where two independent observers performed a frequency and sentiment analysis in parallel to the automated Weblyzard results. Specific articles and user comments on selected key-topics in two major online newspapers in Austria (“Der Standard” and “Die Krone”) were counted and evaluated according to different sentiment categories. The results indicate various weaknesses of the software leading to misinterpretations, and the automated analysis yielded substantially different results compared to the sentiment analysis by the two raters, especially for cynical or irrelevant statements.Our case study highlights the potentials and limits of big-data analyses of media sources compared to those of conventional, quantitative content analysis. The results clearly show that empirical research should always be accompanied by theory-based interpretation and should highlight not only the potentials but also, specifically, the risks and weaknesses of the big data hype to counteract the allure of automated text-based analyses.