Most logging strategies begin with what is easy to collect. Firewalls speak syslog, so firewall logs arrive first and in great volume. The SIEM bill grows, and when an investigation starts, the team finds that the one source it needs was never connected.
A better starting point is the investigation itself. Decide which questions you must be able to answer when something goes wrong, then collect the telemetry that answers them. This article gives a priority order, the reasoning behind it, and the operational details that keep the data usable.
Start from the questions an investigation must answer
Every investigation, whatever the cause, comes back to a small set of questions:
- Who signed in, from where, to what, and was it really them?
- What ran on this device, who started it, and what did it touch?
- What changed in our cloud and SaaS environments, and who changed it?
- What left the organization, and where did it go?
- How far did it spread, and when did it start?
A log source earns its place by answering one of these questions better than anything else you have. If nobody can say which question a source answers, it is a candidate for a cheaper tier or for removal.
This framing also exposes gaps that volume hides. An organization can ingest a very large amount of network data each day and still be unable to say which mailboxes a stolen session opened.
The priority order and the reasoning behind it
The order below assumes an organization with a cloud identity provider, managed endpoints and some public cloud. Adjust it for your architecture, but make the adjustment deliberately.
1. Identity provider sign-in and audit logs
An attacker who holds a valid credential looks like a legitimate user to every other system. The identity provider is often the only place where the difference shows: an unfamiliar device, a new MFA method, a token replayed from another network.
Collect interactive and non-interactive sign-ins, service principal and workload sign-ins, and the audit log of directory changes such as role assignments, application consents and new credentials on applications. This data is also the foundation for any identity security work.
2. Endpoint EDR telemetry
EDR records process creation with command lines, network connections per process, file and registry changes, and script content. It answers "what ran" with a precision that network data cannot match, and it still works when the device is off the corporate network.
Coverage matters more than depth. Every device without an agent is a place you cannot see, so reconcile the agent list against the device inventory regularly.
3. Cloud control plane audit logs
AWS CloudTrail, the Azure Activity log together with Microsoft Entra audit logs, and Google Cloud Audit Logs record who created, changed or deleted resources and permissions. They are low in volume and high in value.
Check the defaults carefully. CloudTrail data events, such as object-level access to S3, and most Google Cloud Data Access logs are off by default and must be enabled for the resources that hold sensitive data. A cloud security review should confirm that every account, subscription and project sends its audit trail to a central location that workload administrators cannot alter.
4. Email
Collect message trace, delivery verdicts, URL click records where available, and mailbox audit events such as inbox rule creation, delegate changes and mail access by applications. Email remains a common entry point. Mailbox audit data is what tells you whether a compromised account was used to read or send mail.
5. DNS and proxy
DNS queries and web proxy logs show where devices tried to go, including devices with no EDR agent. They are the practical way to scope an incident by indicator: which hosts resolved this domain, and when it first happened.
6. Firewall and VPN
VPN authentication and session logs belong at the top of this group, because in practice they are identity events. For firewalls, prioritize connections that cross trust boundaries and traffic to and from the internet. Full internal allow logs are high in volume and are rarely the source that answers a question.
7. Key SaaS audit logs
Collaboration suites, source code hosting, CRM, HR and finance platforms hold the data attackers want, and each has its own audit log. Start with the applications that hold sensitive data or can grant access to other systems. Check the license tier as well, since some vendors restrict audit log access or retention to higher plans.
8. Application logs
Custom applications should log authentication, authorization failures, administrative actions and access to sensitive records, each with a user identifier and a request identifier. They come last only because they need development effort. For a business whose main asset is its own application, they move up the order.
A map from investigation question to log source
| Investigation question | Primary source | Supporting source |
|---|---|---|
| Who signed in, from where, and on what device? | Identity provider sign-in logs | VPN logs, EDR device inventory |
| Were privileges or credentials changed? | Identity provider and cloud audit logs | SaaS administrator audit logs |
| What ran on the device, and what started it? | EDR process telemetry | Script and application logs |
| Which hosts contacted this domain or IP address? | DNS and proxy logs | Firewall logs, EDR network events |
| What was changed in the cloud environment? | CloudTrail, Azure Activity, Google Cloud Audit Logs | Deployment pipeline history |
| Was mail read, forwarded or sent by the attacker? | Mailbox audit logs, message trace | Identity provider sign-in logs |
| Was data taken from a SaaS platform? | SaaS audit logs | Proxy logs |
| What did the attacker do inside our application? | Application logs | Web server and WAF logs |
Use the table as a test. For each row, ask an analyst to answer the question for a named user or host on a date three months ago. The rows where they cannot are your collection backlog.
Retention and cost trade-offs
Log platforms charge for ingestion, for retention, or for both. The way to control cost is to match each source to the storage tier its use requires. Tiering decisions like these are a routine part of SOC modernization.
Hot, searchable storage is for data that detections run against and that analysts query daily. Identity logs, EDR alerts and core endpoint telemetry, cloud audit logs and email events belong here.
Lower-cost or archive storage is for data needed only during investigations or for compliance. It is slower to search and may need a restore or a search job. High-volume firewall, DNS, proxy and verbose endpoint events are typical candidates.
What to keep longer
Keep the low-volume, high-value sources for the longest period: identity sign-in and audit logs, cloud control plane logs, and SaaS and mailbox audit logs. Intrusions are sometimes found long after they began, and these are the sources that reconstruct the early stages. They are also inexpensive to retain because they are small.
Set retention from three inputs: regulatory or contractual requirements, how far back your own investigations have needed to look, and the native retention of each source. Several cloud and SaaS platforms keep audit data for a short default period. If you do not export it, it is gone when you need it.
Reduce volume before it reaches the hot tier. Filter events that answer no question, drop unused fields, and summarize repetitive network flows. Record every filter, because a filter is also a blind spot.
Log health monitoring
A log source that stops sending is a loss of visibility. Handle it as an incident with an owner and a deadline. The cause may be an expired API key or a decommissioned collector. It may also be an attacker disabling an agent or stopping a cloud trail.
Monitor at least the following:
- Silence. No events from a source, or from an individual device, for longer than its normal interval. Set the threshold for each source, since a domain controller and a quarterly batch system behave differently.
- Volume change. A sudden drop or spike against the source's own baseline.
- Ingestion delay. The gap between event time and arrival time. A detection that looks back one hour cannot see events that arrive three hours late.
- Parsing failures. A vendor update that renames a field can break detections without stopping the flow.
- Coverage. Compare the device inventory with the list of devices that are actually reporting.
- Tampering events. Alerts for stopped trails, deleted diagnostic settings, disabled agents and cleared event logs.
Send health alerts to the same queue as security alerts. A separate mailbox that nobody reads is the usual reason a silent source goes unnoticed.
Where to start
- Write down the investigation questions that matter to your organization, and mark the ones you can answer today with evidence.
- Confirm that identity provider, EDR and cloud audit logs are complete across every tenant, account and device group before adding any new source.
- Check the default retention of each cloud and SaaS audit log, and export the ones that expire too early.
- Move high-volume, low-use sources to a cheaper tier and record the decision.
- Build a silence alert for each log source and assign it an owner. An incident readiness exercise is a good way to test whether the data holds up.