Context sensitive anomaly monitoring of process control flow to detect mimicry attacks and impossible paths

Haizhi Xu, Wenliang Du, Stephen J Chapin

Research output: Contribution to journalArticle

11 Scopus citations


Many intrusions amplify rights or circumvent defenses by issuing system calls in ways that the original process did not. Defense against these attacks emphasizes preventing attacking code from being introduced to the system and detecting or preventing execution of the injected code. Another approach, where this paper fits in, is to assume that both injection and execution have occurred, and to detect and prevent the executing code from subverting the target system. We propose a method using waypoints: marks along the normal execution path that a process must follow to successfully access operating system services. Waypoints actively log trustworthy context information as the program executes, allowing our anomaly monitor to both monitor control flow and restrict system call permissions to conform to the legitimate needs of application functions. We describe our design and implementation of waypoints and present results showing that waypoint-based anomaly monitors can detect a subset of mimicry attacks and impossible paths.

Original languageEnglish (US)
Pages (from-to)21-38
Number of pages18
JournalLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
StatePublished - 2004



  • Anomaly detection
  • Context sensitive
  • Control flow monitoring
  • Impossible paths
  • Mimicry attacks
  • Waypoint

ASJC Scopus subject areas

  • Biochemistry, Genetics and Molecular Biology(all)
  • Computer Science(all)
  • Theoretical Computer Science

Cite this