This page contains selected talks, papers, and other publications.
- Benjamin Holland. Program Analysis for Cybersecurity. US Cyber Challenge Summer Bootcamps (USCC 2017), Illinois, Delaware, and Utah, July 2017.
Paper: [Abstract] [Materials]
- Benjamin Holland, Ganesh Ram Santhanam, Suresh Kothari. Transferring State-of-the-art Immutability Analyses: Experimentation Toolbox and Accuracy Benchmark. The 10th IEEE International Conference on Software Testing, Verification and Validation (ICST 2017), Tokyo, Japan, March 2017.
Paper: [Abstract] [Paper] [Toolbox] [Benchmark]
Immutability analysis is important to software testing, verification and validation (V&V) because it can be used to identify independently testable functions without side-effects. Existing tools for immutability analysis are largely academic prototypes that have not been rigorously tested for accuracy or have not been maintained and are unable to analyze programs written in later versions of Java. In this paper, we re-implement two prominent approaches to inferring the immutability of an object: one that leverages a points-to analysis and another that uses a type-system. In addition to supporting Java 8 source programs, our re-implementations support the analysis of compiled Java bytecode. In order to evaluate the relative accuracy, we create a benchmark that rigorously tests the accuracy boundaries of the respective approaches. We report results of experiments on analyzing the benchmark with the two approaches and compare their scalability to real world applications. Our results from the benchmark reveal that points-to based approach is more accurate than the type inference based approach in certain cases. However, experiments with real world applications show that the points-to based approach does not scale well to very large applications and a type inference based approach may offer a scalable alternative.
- Benjamin Holland. Exploring the space in between bugs and malware. Iowa State University Cybersecurity Seminar Series, Ames, Iowa, November 2016.
Talk: [Abstract] [Flier] [Video]
We live in an age of software problems with catastrophic consequences. An extra goto in Apple's SSL implementation compromised certificate checks for the better part of a year. An erroneous integer conversion in the Ariane 5 launch destroyed the European Space Agency rocket and its cargo valued at 500 million dollars. Often the problem is just a few lines of code and looking for it is like searching for a needle in a haystack, but without knowing what a needle looks like. Moreover the problems are often so subtle that it is difficult to tell if the problem is intentionally malicious or an honest mistake. The traditional approach to bug and malware detection fails to detect novel attacks or discover new classes of bugs. To make matters worse, both problems can remain dormant and can easily evade testing. In this talk we critically explore the challenges involved in bug and malware detection. To explore concepts further we leverage a framework called JReFrameworker for manipulating the Java runtime environment to develop managed code rootkits.
- Suresh Kothari, Benjamin Holland. Discovering Information Leakage Using Visual Program Models. MILCOM 2016, Baltimore, Maryland, November 2016.
Tutorial: [Abstract] [Materials]
This tutorial is about new genera of information leakage vulnerabilities, far more difficult to detect than the vulnerabilities that have previously dominated the software security landscape. We will survey attacks that have exploited information leakage vulnerabilities to steal sensitive information. We will show how to discover information leakage vulnerabilities using techniques and tools for visual modeling of software from our research on two high-profile DARPA programs, the Automated Program Analysis for Cybersecurity (APAC) and Space/Time Analysis for Cybersecurity (STAC).
The tutorial goals are:
- Provide broad knowledge and the key concepts about information leakage threats.
- Teach interactive tools to create visual models to analyze Java bytecode.
- Provide hands-on experience of applying visual models with interactive visualization to audit an application for information leakage threats.
- Benjamin Holland, Ganesh Ram Santhanam, Payas Awadhutkar, and Suresh Kothari. Statically-informed Dynamic Analysis Tools to Detect Algorithmic Complexity Vulnerabilities. The 16th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM 2016), Raleigh, North Carolina, October 2016.
Paper: [Abstract] [Paper] [Slides] [Video] [SID Tools]
Algorithmic Complexity (AC) vulnerabilities can be exploited to cause a denial of service attack. Specifically, an adversary can design an input to trigger excessive (space/time) resource consumption. It is not possible to build a fully automated tool to detect AC vulnerabilities. Since it is an open-ended problem, a human-in-loop exploration is required to find the program loops that could have AC vulnerabilities. Ascertaining whether an arbitrary loop has an AC vulnerability is itself difficult, which is equivalent to the halting problem.
This paper is about a pragmatic engineering approach to detect AC vulnerabilities. It presents a statically-informed dynamic (SID) analysis and two tools that provide critical capabilities for detecting AC vulnerabilities. The first is a static analysis tool for exploring the software to find loops as the potential candidates for AC vulnerabilities. The second is a dynamic analysis tool that can try many different inputs to evaluate the selected loops for excessive resource consumption. The two tools are built and integrated together using the interactive software analysis, transformation, and visualization capabilities provided by the Atlas platform.
The paper describes two use cases for the tools, one to detect AC vulnerabilities in a Java bytecode and another to use the tools in an undergraduate algorithm class for students to perform experiments to learn different aspects of algorithmic complexity.
- Suresh Kothari, Benjamin Holland. Managing Complexity, Security, and Safety of Large Software. Global Initiative of Academic Networks (GIAN), Jaipur, India, September 2016.
Seminar: [Abstract] [Materials]
Software malfunctions can have catastrophic consequences and enormous costs. The error-prone and laborious manual practices of developing and maintaining large software must change to cope with ever increasing complexity of software, and the enormous safety and security challenges it poses. Formal verification is not a practical alternative for large software; it is riddled with problems of low accuracy and high computational complexity. The need for automation in software engineering is undoubted, however, a human is indispensable to reason about complex software. With this background, we have conducted two decades of research on human-in-loop automation (HLA) as a pragmatic alternative to manage complexity, safety, and security of large software systems in the cyber-physical world.
This course will introduce the HLA technology developed through our research, with unprecedented capabilities to model, analyze, visualize, verify, and transform large software in multiple languages. The HLA technology incorporates a graph database platform with a powerful query language to extract knowledge from software and build visual models to solve complex problems through automation guided by human reasoning. The HLA technology has been validated by applying it to: automatic parallelization of climate model software, model-based software development for critical avionics and automobile control systems, safety verifications of the Linux operating system, and cybersecurity challenges posed by Defense Advanced Research Projects Agency (DARPA) research programs. More than 300 companies including all major avionics and automobile companies use the HLA tools developed by EnSoft, the company founded on our research.
This course will provide a solid foundation on fundamentals of software that are applicable across all programming languages. With examples of real-world software problems, we will elaborate how this fundamental knowledge can be applied using HLA to cope with software complexity, safety, and security. We will present HLA as a modeling and problem-solving activity. Starting with a HLA model for systematic debugging, the course will show how to gradually advance to build HLA models for verifying complex software such as the Linux kernel. As in physics and mathematics, rigorous problem solving will be taught with models as powerful abstractions. We will use the Atlas platform from EnSoft to build visual models of software. Atlas stores program semantics in a graph database, and provides interactive and programming interfaces to query, model, and visualize software.
A strong laboratory component will include demonstrations, interactive experiments, and programming exercises. The labs will include challenging problems from real-world software in C, Java, and Java byte code. The participants will get a first-hand experience of how to implement HLA techniques through Atlas and perform analysis tasks in few minutes. These will include examples of difficult analysis tasks for which automated analysis is intractable, manual effort is prohibitively high, and thus HLA is the only practical solution.
The lecture notes, exercises, and the HLA tools for conducting the labs will be available to the participants for free academic use. The course will provide transformative ideas and introduce tools to implement them. It will enable academic participants to transform their research and education to focus on real-world problems of large software. It will impel new thinking among industry and government participants to innovate application-specific HLA practices and tools that can save significant time, money and effort with unprecedented capabilities to achieve safety and security in their software systems.
- Suresh Kothari, Benjamin Holland. Learn to Build Automated Software Analysis Tools with Graph Paradigm and Interactive Visual Framework. The 31st IEEE/ACM International Conference on Automated Software Engineering (ASE 2016), Singapore, September 2016.
Tutorial: [Abstract] [Materials]
Software analysis has become complex enough to be intimidating to new students and professionals. It can be difficult to know where to start with over three decades of staggering research in data and control flow analyses and a plethora of analysis frameworks to choose from, ranging in maturity, support, and usability. While textbooks, surveys and papers help, nothing beats the personal experience of implementing and experimenting with classic algorithms.
With support from DARPA, we have developed a graph paradigm enabled with an interactive visual framework to implement and experiment with software analysis algorithms. Parsed programs along with pre-computed data and control flows are stored as a graph database so that analyzers with varying degrees of accuracy and scalability tradeoffs can be easily implemented using a high-level query language. The graphical as well as textual composition of queries, interactive visualization, and the 2-way correspondence between the code and its graph models are integrated through a platform called Atlas. With this machinery, the implementation and visualization effort is reduced as much as 10 to 50 fold, making it much easier to learn about and do research on software analysis algorithms with applications to software safety and security. The tutorial will provide the necessary background, including implementation of widely used algorithms. Participants will learn to prototype several algorithms in a short timeframe.
- Benjamin Holland (daedared). Developing Managed Code Rootkits for the Java Runtime Environment. DEFCON 24, Las Vegas, Nevada, August 2016.
Talk: [Abstract] [Slides] [Video] [JReFrameworker] [Demos 1, 2, 3, 4]
Managed Code Rootkits (MCRs) are terrifying post-exploitation attacks that open the doors for cementing and expanding a foothold in a target network. While the concept isn't new, practical tools for developing MCRs don't currently exist. Erez Metula released ReFrameworker in 2010 with the ability to inject attack modules into the C# runtime, paving the way for MCRs, but the tool requires the attacker to have knowledge of intermediate languages, does not support other runtimes, and is no longer maintained. Worse yet, the "write once, run anywhere" motto of managed languages is violated when dealing with runtime libraries, forcing the attacker to write new exploits for each target platform.
This talk debuts a free and open source tool called JReFrameworker aimed at solving the aforementioned challenges of developing attack code for the Java runtime while lowering the bar so that anyone with rudimentary knowledge of Java can develop a managed code rootkit. With Java being StackOverflow's most popular server side language of 2015 the Java runtime environment is a prime target for exploitation. JReFrameworker is an Eclipse plugin that allows an attacker to write simple Java source to develop, debug, and automatically modify the runtime. Best of all, working at the intended abstraction level of source code allows the attacker to "write once, exploit anywhere". When the messy details of developing attack code are removed from the picture the attacker can let his creativity flow to develop some truly evil attacks, which is just what this talk aims to explore.
- Benjamin Holland. Stealing Web Browser Cookies. Iowa State University Friday at Noon Activity, Ames, Iowa, April 2016.
Talk: [Abstract] [Flier] [Slides] [Video] [CookieMonster]
After you log into a website your web browser is given a "cookie". Until you log out, your identity is your possession of that cookie. It's no wonder attackers love to steal your cookies. Let's take a look at how attackers steal from the cookie jar, what they can do with them, and how you can protect yourself.
- Suresh Kothari, Benjamin Holland. Computer-aided Collaborative Validation of Large Software. The 30th IEEE/ACM International Conference on Automated Software Engineering (ASE 2015), Lincoln, Nebraska, November 2015.
Tutorial: [Abstract] [Materials]
Neither manual nor totally automated discovery of software vulnerabilities is practical. Manual discovery requires extremely laborious work by highly skilled software analysts and totally automated discovery is riddled with intractable problems.
This tutorial introduces a novel practical approach for machine-enabled human-in-the-loop discovery of software vulnerabilities, and is based on "amplifying human intelligence" rather than trying to replace human intelligence. The approach is supported by a suite of tools with unique capabilities that enable human analysts to quickly identify and understand the relevant parts of large software and perform "what-if experiments" in order to discover highly sophisticated vulnerabilities. These tools are advanced through large Defense Advanced Research Projects Agency (DARPA) projects and their effectiveness has been demonstrated for discovering sophisticated malware challenges.
- Jeremías Sauceda, Benjamin Holland, Suresh Kothari. Visual Models to Solve Hard Problems at the Intersection of Cybersecurity and Software Reliability. Invited Brown Bag Presentation at Rockwell Collins, Cedar Rapids, Iowa, November 2015.
- DARPA Research
- Unify cybersecurity and software reliability – how and why?
- Solving hard problems with visual models
- Android App Audits
- Tool Demonstration
- Suresh Kothari, Benjamin Holland. Hard Problems at the Intersection of Cybersecurity and Software Reliability. The 26th IEEE International Symposium on Software Reliability Engineering (ISSRE 2015), NIST, Gaithersburg, Maryland, November 2015.
Tutorial: [Abstract] [Materials]
This tutorial is aimed at the audience interested in knowing how software reliability and cybersecurity converge in terms of intrinsic hard problems, and how that knowledge can be useful for advancing the research and practice in both fields. This tutorial is based on our research in three Defense Advanced Research Projects Agency (DARPA) projects and our practical experience of applying the research. The tutorial will provide succinct understanding of the "hardness" through representative problems and by introducing a programming language agnostic notion of an intrinsic hardness spectrum derived from fundamental impediments to detecting vulnerabilities accurately. About 60% of the tutorial will be demonstrations to elaborate on the hardness spectrum and its practical applicability. The representative problems will pertain to reliability issues for operating system kernels and malware attacks on Android apps. We will introduce the use of a powerful program comprehension tool to derive the hardness spectrum by mapping the Java, C, and Java bytecode to high-level entities that reveal the inner workings of complex software.
- Suresh Kothari, Benjamin Holland. Practical Program Analysis for Discovering Android Malware. MILCOM 2015, Tampa, Florida, October 2015.
Tutorial: [Abstract] [Materials]
The growing threat of malware in embedded systems and the possibility of adversaries crafting one-of-a-kind sophisticated malware as a catastrophic cyberweapon makes malware detection a high priority topic for advanced research, college education, and professional training. There is a need for automated detection tools for commercial applications as well as a need for sophisticated apparatus to discover evasive malware targeted at defense applications. With those needs in mind, this tutorial will show the participants how to effectively deploy program analyses for cybersecurity.
The tutorial will be filled with interesting demonstrations of practical techniques, their applicability and limitations, and the underlying formal framework for future advances. We will discuss a novel and easy-to-understand graph paradigm of program analysis as the backbone of the framework. We will introduce Atlas, a platform designed to deploy the graph paradigm effectively. We will show how Atlas makes it easy to develop automated tools by taking away the burden of programming low-level program analysis constructs. We will demonstrate the Android Security Toolbox, an Atlas plug-in, as a sophisticated apparatus we developed through the DARPA APAC program.
The participants will get a hands-on experience in the powerful applicability of the graph paradigm through Atlas by observing its use to perform visual graph interactions or write small graph traversal programs to perform analyses that would otherwise take days of conventional programming. Researchers, college professors, and professional practitioners will find that they can build on the tutorial material and incorporate the graph program analysis paradigm to foster their special interests in research, teaching, or professional practice.
- Benjamin Holland, Amber Aldrich. Homebrewing for Hackers. Derbycon 5.0, Louisville, Kentucky, September 2015.
Talk: [Abstract] [Slides] [Video]
It has been a few years now since int eighty told us to drink all the booze and hack all the things. Well there are plenty of things left to hack, but the cost of booze is really adding up. Come learn how you can cheaply and easily brew your own booze from the many varieties of beer to a Viking mead. We will even touch a little on the legal and historical side of homebrewing. Did you know it's legal to brew your own in all 50 states as long as you don't sell it? Well you will when we give FREE BEER* to a few lucky attendees because we can't legally sell it to you! That's right FREE as in BEER. Finally we are open sourcing a markdown based homebrew log built on the Git version control system so you can publicly record your beer measurements and automatically calculate the alcohol content!
*Some molecular assembly required. Must be 21 years of age. Not enough beer for everyone.
- Benjamin Holland. There’s a hole in my bucket, dear Liza - Examining side channel leaks in web apps. OWASP Ames, Ames, Iowa, August 2015.
Talk: [Abstract] [Slides] [Video]
Think twice before you optimize that code! You might just give away the farm. Side channel attacks were traditionally used to reverse engineer cryptographic hardware circuits using power analysis, but more recently timing information is being used to deduce the sensitive inner workings of software. The steady stream of side channel exploits coming out academia and the security community continue to demonstrate the seriousness of the problem and DARPA's current Space/Time Analysis for Cybersecurity (STAC) program indicates that we need a solution now. Let's take a look at a few real examples of information leakage through side channel attacks in web apps and learn to spot them together. If there's a hole in your bucket, then fix it, dear Henry.
- Benjamin Holland, Tom Deering, Suresh Kothari, Jon Mathews, Nikhil Ranade. Security Toolbox for Detecting Novel and Sophisticated Android Malware. The 37th International Conference on Software Engineering (ICSE 2015), Firenze, Italy, May 2015.
Paper: [Abstract] [Paper] [Video]
This paper presents a demo of our Security Toolbox to detect novel malware in Android apps. This Toolbox is developed through our recent research project funded by the DARPA Automated Program Analysis for Cybersecurity (APAC) project. The adversarial challenge ("Red") teams in the DARPA APAC program are tasked with designing sophisticated malware to test the bounds of malware detection technology being developed by the research and development ("Blue") teams. Our research group, a Blue team in the DARPA APAC program, proposed a "human-in-the-loop program analysis" approach to detect malware given the source or Java bytecode for an Android app. Our malware detection apparatus consists of two components: a general-purpose program analysis platform called Atlas, and a Security Toolbox built on the Atlas platform. This paper describes the major design goals, the Toolbox components to achieve the goals, and the workflow for auditing Android apps. The accompanying video illustrates features of the Toolbox through a live audit.
- Benjamin Holland. Learning to Think Like a Hacker. Hosted at Multiple Events, Ankeny/Ames, Iowa, April 2015.
Workshop: [Abstract] [Slides] [Voicemail Victim]
This workshop was presented at multiple venues: Des Moines Area Community College's Discover Engineering Days, Iowa State University's Society of Women Engineering University, and Iowa State University's IT Olympics. The material was designed for high school level students with the goal of developing critical thinking skills and sparking an interest in engineering careers. During the IT Olympics this material was used to demonstrate to high school educators a way to challenge students to think critically.
Student Session Description: Computer hackers are a dark mysterious group both feared and revered in popular culture. What makes a hacker different? Come learn how anyone can be a hacker as we get our hands dirty attacking physical, digital, and human systems.
Educator Session Description: A workshop that discusses teaching critical thinking, curiosity, and experimentation to students through computer security. The workshop involves hands on labs for disabling tamper evident devices, voicemail hacking, and social engineering, as well as legal and ethical issues in security.
Benjamin Holland, Suresh Kothari, Jeremías Sauceda. Comprehension-Driven Program Analysis for Cybersecurity. Summary and demonstration of Iowa State University performer progress during phase two of DARPA’s Automated Program Analysis for Cybersecurity (APAC) program to attending government representatives. DARPA Headquarters, Arlington, Virgina, October 2014.
Suresh Kothari, Akshay Deepak, Ahmed Tamrawi, Benjamin Holland, Sandeep Krishnan. A “Human-in-the-loop” Approach for Resolving Complex Software Anomalies. The 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC 2014), San Diego, California, October 2014.
Paper: [Abstract] [Paper]
Automated static analysis tools are widely used in identifying software anomalies, such as memory leak, unsafe thread synchronization and malicious behaviors in smartphone applications. Such anomaly-prone scenarios can be bifurcated into: "ordinary" (analysis requires relatively simple automation) and "complex" (analysis poses extraordinary automation challenges). While automated static analysis tools can resolve ordinary scenarios with high accuracy, automating the analysis of complex scenarios can be very challenging and, at times, infeasible. Even when feasible the cost for full automation can be exorbitant: either in implementing the automation or in sifting through the large number of erroneous results manually. Instead, we appeal for a "Human-in-the-loop" approach called "Amplified Reasoning Technique" (ART). While some of the existing approaches do involve human in the analysis process, the roles played by man and machine are mainly segregated. Whereas, ART puts man and machine in a "loop" in an interactive and visualization-based fashion. This paper makes an attempt to convince its readers to make their analysis of software anomalies ART-based by presenting real-world case studies of complex anomalies and how an ART based approach can be very effective in resolving them. The case studies highlight the desired characteristics of an ART based tool and the type of role it plays in amplifying human intelligence.
- Benjamin Holland, Suresh Kothari. A Bug or Malware? Catastrophic consequences either way. Derbycon 4.0, Louisville, Kentucky, September 2014.
Talk: [Abstract] [Slides] [Video]
We live in an age of software problems with catastrophic consequences. An extra goto in Apple's SSL implementation compromised certificate checks for the better part of a year. An erroneous integer conversion in the Ariane 5 launch destroyed the European Space Agency rocket and its cargo valued at 500 million dollars. Often the problem is just a few lines of code and looking for it is like searching for a needle in the haystack. Moreover the problems are often so subtle that it is difficult to tell if the problem is intentionally malicious or an honest mistake. The traditional approach to malware detection fails to detect such catastrophic problems. To make matters worse, the problem can remain dormant and can easily evade testing. The recently exposed Heartbleed problem in OpenSSL has existed since 2011. It is an open challenge to discover these subtle but catastrophic problems in software. In this talk, Iowa State University researchers involved with DARPA's Automated Program Analysis for Cybersecurity (APAC) project will discuss their approach to address this challenge. This approach enables a unique combination of automated software analysis and human intelligence. The approach will be concretely demonstrated by its use to detect subtle problems in Android applications.
- Benjamin Holland, Yong Guan. Revealing Privacy in Online Social Networks - An exercise in revealing private friends. Unpublished, July 2012.
Paper: [Abstract] [Paper]
With rising privacy concerns and the widespread adoption of social media platforms, it becomes necessary to examine how much information is still exposed after privacy protection mechanisms are enabled. Malicious attackers and investigators looking to utilize social media often face privacy and platform restrictions that serve both to protect the privacy of individuals and to protect the economic livelihood of the social media platform. This work focuses on revealing private social contacts as a first step to a larger reconnaissance of a target Online Social Network (OSN) profile. In many OSNs the friends that are hidden when a user enables privacy protection mechanisms may still be discovered by visiting the profiles of the target’s friends and searching for public references to the target account. For large OSNs such as Facebook it would be unfeasible to search large portions of the network to discover friends of the target. In this work, we review existing Social Network Analysis (SNA) research to examine how it can be exploited to efficiently discover a majority of private friends. As our contribution, we propose a greedy search algorithm for enabling efficient discovery of private friends on social networking sites, which has shown a 33.6% to 41.6% improvement over a standard breadth-first search on synthetically generated social networks as well as real-world social networks.
- Benjamin Holland. Enabling Open Source Intelligence (OSINT) in private social networks. Iowa State University, Ames, Iowa, May 2012.
Master’s Thesis: [Abstract] [Thesis]
Open Source Intelligence (OSINT) has been widely acknowledged as a critical source of valuable and cost efficient intelligence that is derived from publicly available sources. With the rise of prominent social media platforms such as Facebook and Twitter that record and expose a multitude of different datasets, investigators are beginning to look at what social media has to offer the Intelligence Community (IC). Some major obstacles that OSINT analysts often face are privacy and platform restrictions that serve both to protect the privacy of individuals and to protect the economic livelihood of the social media platform. In this work we review existing social networking research to examine how it can be applied to OSINT. As our contribution, we propose a greedy search algorithm for enabling efficient discovery of private friends on social networking sites and evaluate its performance on multiple randomly generated graphs as well as a real-world social network collected by other researchers. In its breadth, this work aims to provide the reader with a broader understanding of OSINT and key concepts in social network analysis.