Five open data formats that every cybersecurity operations team needs to know about

By Vaughan Shanks

Faster cybersecurity operations is underpinned by interoperability.

Security teams are often measured on their ability to move quickly, with efficacy metrics couched in terms like mean time to detect and contain a threat, mean time to recover compromised systems, and average dwell time for attackers.

All require care and due process, but at the same time a certain level of expediency is demanded.

Teams are able to move faster when all the parts of security operations are in sync. Tooling plays a key role in supporting operational security teams to perform their work more effectively, but security setups are historically complex. Case-in-point: Gartner recently found 75% of teams are trying to slim the number of security vendors they work with to reduce complexity and improve risk posture. This is up from 29% in 2020.

Environments will still be best-of-breed, just with fewer vendors. There are good reasons for wanting this. A best-of-breed approach ensures teams have access to best-in-category tooling at the best price. No single vendor can do every part of cybersecurity operations well, and consolidation brings the risk of vendor lock-in.

The reason the number of tools, and vendors, involved in best-of-breed environments is considered a problem is due to the difficulty in connecting all of these systems together. That task is traditionally left to the security operations team itself to try to establish a common ontology, or structure, for data collection across all these systems. That means a lot of challenging integration work and software development.

The problem has multiplied in recent years with the realisation that security is a team sport. Whole industries now unite – such as through information sharing and analysis centre (ISAC) community structures – to exchange intelligence and collectively impose costs on attackers by exposing their infrastructure and methods. Shareable tradecraft is increasingly critical in security contexts but is made much more difficult when data is collected in piecemeal and proprietary formats and is difficult to share.

Security teams need simpler ways to understand the vast IT estates they protect. They need standard ways to collect and ingest data into a central log management store to improve their visibility and reaction time. They need systems that automatically recognise what data has been collected and what to do with it: how to index it and make it queryable, how to understand the vulnerabilities and security control posture of their organisation, and how to share threat information and response processes with other security teams.

If that happens ‘magically’ without people having to hop between different screens, or without having an engineer wire it all together over many months and thousands of dollars in consulting fees, then that’s a win for security teams everywhere.

What winning looks like

Vendors have a role to play in relieving security teams of this extra effort, and they increasingly are by adopting common data formats as standards.

There are several data standards efforts that are leading the way in making the job of operational security teams simpler. Five standards that all security teams should be aware of are STIX, OCSF, OSCAL, SPDX, and CACAO.

Structured Threat Information eXpression (STIX), which comes under the auspices of The Organization for the Advancement of Structured Information Standards – better known as OASIS Open – is one such data format and success story for interoperability. As it is designed to enable the exchange of cyber threat intelligence, STIX is particularly useful for teams and organisations involved in sharing communities and forums.

Another OASIS Open specification that is gaining traction is Collaborative Automated Course of Action Operations (CACAO). CACAO allows incident response playbooks to be shared between security operations teams using a machine-readable JSON format. CACAO playbooks may include sequential and parallel actions, decisions, loops, and error handling. The steps in the CACAO playbooks may be manual or automated, with variables passed between steps. Additional variables in the playbook metadata allow a playbook to be customised to a particular environment.

More recently, the Open Cybersecurity Schema Framework (OCSF), announced at Black Hat USA 2022 and backed by some of the largest technology vendors in the world, has also shown considerable promise. OCSF is notable because some of its backers are organisations that, in the past, would not have produced easily interoperable tools. By standardising the way data is defined and logged in the cyber observable space, security teams using tools that are compatible with the OCSF standard can be confident that field names and timestamp formats, for example, are the same and aligned across all tools.

For security posture management, the Open Security Controls Assessment Language (OSCAL), published by NIST, is a convenient format for representing formal lists of security controls. This format has been used to publish the US NIST Security and Privacy Controls for Information System and Organizations (SP 800-53r5), the US Federal Risk and Authorization Management Program (FedRAMP), and more recently, the Australian Information Security Manual (ISM). OSCAL has schemas in JSON, YAML, and XML, and allows catalogs of security controls to be both human and machine readable.

For posture management on a more tactical level, the Software Package Data Exchange (SPDX) is an open standard for transmitting a software bill of materials (SBOM). SBOMs were mandated for US Federal agencies by a Presidential Executive Order in May 2021, to ensure that agencies could track the provenance of the software components included in Federal information systems. SPDX was adopted as an international standard, ISO/IEC 5962:2021, in August 2021. The ability to receive, store, and query SPDX enables a security team to rapidly find vulnerable dependencies in the IT environment of their organisation.

Seamless integration through the adoption of standardised data formats can save security teams a lot of friction and help them to react faster, minimising time to triage, analyse and contain threats. That has flow-on impacts, reducing the window of time that an attacker has to conduct internal reconnaissance and lateral movement in a network. Ultimately, if the window of opportunity is reduced enough by faster response, it prevents an attacker from achieving their actions on objective.