DaC Reference

================================ Detections as Code


Elastic Reference
_______________
Managing Elastic Security Detection Rules Using DaC

If you’re new to this repo and looking to get started, see the DaC Quick Start Guide to get set up and running. Then review the full DaC Concept and Workflows and the rest of the documentation in this guide for more information on concepts and workflows that you may want to use when setting up your CI/CD.

Also, please see the video in the End-to-End Reference guide for an example of what this can look like in practice; DaC between the detection rules repo and Kibana with version control and automated syncing, testing, etc. via CI/CD provided by GitHub.

For additional reference, see the content within these slides.

💡 Note: This documentation is focused on the Detection Rules DaC reference implementation, but the concepts and workflows can be applied to your own implementation and content as well. Other tools, such as Elastic’s Terraform provider, can also be used to implement DaC for Elastic Security; those approaches are outside the scope of this guide.

Documentation

• DaC Quick Start Guide
  • Prerequisites
  • 1. Set up the detection-rules repo
  • 2. Custom rules directory
  • 3. Connecting to the Elastic stack
  • 4. Developing rules
  • 5. Unit testing
  • 6. Custom configuration (_config.yaml)
  • 7. Schema validation
  • 8. Exceptions and actions
  • 9. Syncing with Kibana
  • 10. Version locking (optional)
  • Known limitations and FAQ links
• DaC Concept and Workflows
  • Overview
  • Applying DaC Principles to Rules in Elastic
  • Managing Detection Rules beyond Elastic Security with VCS
  • How to use this reference
  • Overall Elastic DaC Diagram
  • Core Components of DaC Delineation
  • Governance Models of DaC Delineation
  • Best Practices
• Core components and Governance Models of DaC
  • Core components of Detections as Code
  • Governance Models Dictating the Core Components of DaC
  • Sub-Components and Options
  • Detailed Considerations of the Different Governance Models
• Internals of the detection-rules Repo and How it is Used Internally
  • Overview of Elastic Detection Rules Repo and Usage
  • Getting Started with the Repo
  • CLI
  • Unit tests
  • Schemas and Data Structures
  • Incorporating Custom Rules
  • Creating Detection Rules
  • Rule Versioning
  • Exceptions and Actions
  • Unit testing
  • Rule Schema Validation
  • Detection Logic Validation
• Core Component: Managing Detection Rules in a VCS
  • Overview
  • Considerations
  • Sub-Component 1: Elastic Detection Rules Repo and Usage
  • Sub-Component 2: Incorporating Custom Rules
  • Sub-Component 3: Creating Detection Rules
  • Sub-Component 4: Rule Versioning
  • Sub-Component 5: Exceptions and Actions
  • Sub-Component 6: Unit testing
  • Sub-Component 7: Rule Schema Validation
  • Sub-Component 8: Detection Logic Validation
• Core Component: Syncing Rules and Data from VCS to Elastic Security
  • Overview
  • Considerations
  • Sub-Component 1: Deploying via Direct HTTP Request API Calls
  • Sub-Component 2: Deploying via CI/CD
  • Sub-Component 3: Pre-Production Testing
• Core Component: Managing Detection Rules in Elastic Security
  • Overview
  • Considerations
  • Sub-Component 1: Source of Truth
• Core Component: Syncing Rules and Data from Elastic Security to VCS
  • Overview
  • Considerations
  • Sub-Component 1: Exporting Rules via Direct HTTP Request API Calls
  • Sub-Component 2: Exporting Rules via CI/CD
  • Sub-Component 3 (optional): Unit Testing Rules via CI/CD
• E2E Reference
  • Quick Start Slides
  • Demo Video
  • Quick Start Example Detection Rules CLI Commands
• Feedback and Resources
  • Feedback
  • Updates and Next Steps
  • Resources
• Known Issues and Limitations Using Elastic’s detection-rules DaC Approach
  • Intermittent Exception List Kibana Import Issue
  • Rule Size Limitations
  • Schema Validation Support
  • No Direct Support
  • Potential Upcoming Enhancements
• About this guide and project
  • Licensing and Use
• Frequently Asked Questions
  • Q1: What is Detections as Code (DaC)?
  • Q2: How does DaC benefit my security team?
  • Q3: Can I integrate DaC with my existing CI/CD pipelines?
  • Q4: How can setup and use an API Key for authentication with the stack?
  • Q5: What are the prerequisites for adopting DaC with Elastic?
  • Q6: How do I start using DaC in my organization?
  • Q7: Are there any resources available for learning more about DaC?
  • Q8: How can I backup my rules prior to overwriting rules in Kibana?
  • Q9: How can I contribute to the DaC methodology or detection-rules repo?
  • Q10: Where is the best place to add custom code?
  • Q11: How should exception & action files be deployed to Kibana?
  • Q12: Is there a way to run the unit tests only on custom_rules_dir?
  • Q13: I want to use export-rules-from-repo in CI/CD to convert a new/modified rule from TOML to JSON and push the JSON to Kibana API. What is the best way to do that?
  • Q14: I want to fork the Detection Rules repo on GitHub, but I want to keep my fork private. What is the best way to do that?
  • Q15: When using custom schemas, do I have to declare all fields or only those that are not part of the ecs?