Detecting Suspicious Sign-Ins using AI: Our Improved Solution for On-Premises Environments
By: Julik Keijer, Jair Santanna, and Christiaan Ottow
Introduction
In this blog, we discuss a security gap we’ve identified in coverage for on-premises environments related to sign-in events. To address this, we now offer enhanced protection by leveraging our own ML/AI models for clients who maintain an on-premises Active Directory footprint.
Monitoring suspicious sign-ins is a critical aspect of cyber security. While not every unusual login attempt signals malicious intent, these events can reveal potential breaches. The challenge? Spotting the difference before it's too late.
Now, AI offers a valuable solution by helping cyber security teams quickly and precisely detect, correlate, and respond to suspicious actions. Advanced AI models can analyse patterns and detect anomalies in attempted logins. For instance, a single suspicious sign-in might seem benign on its own. However, when paired with anomalies like unusual data access or the registration of MFA devices, it can reveal a significant threat. Here’s how we developed a solution to aid us in making these correlations, so we can proactively identify and mitigate potential breaches in on-premises environments.
Enhancing Cloud Detection with Microsoft Tools
At Northwave, we use a suite of advanced tools, including Microsoft’s security solutions, to protect our clients. Microsoft Entra ID (formerly Azure AD), a cloud-based identity and access management service, provides “risky sign-in events” [1] flagged by its threat intelligence and machine learning algorithms as potentially malicious or unauthorised.
In a previous blog, we detailed how our self-developed ML/AI models outperform Microsoft’s risky sign-ins and can be integrated with other cloud-based detections to strengthen overall security coverage[2]. While this combination is highly effective, we identified a gap in coverage for on-premises environments. To fill this gap, we developed our own suspicious sign-in detection system, powered by ML/AI, to extend robust protection to on-premises Active Directory (AD) environments.
Extending Cloud Capabilities to On-Premises AD
We apply the same comprehensive approach to on-premises environments as we do in the cloud. Suspicious activity is detected across three key pillars: identity, network, and endpoint. Using tools like our custom developed rules based on Windows Security Events, network sensors, and endpoint detection and response (EDR) systems, we gather various detections with differing levels of suspiciousness. These events are then correlated to identify malicious patterns.This method is particularly effective for early-stage events in the kill chain or when adversaries generate multiple alerts through “loud” activities. However, challenges arise when high-impact alerts lack associated indicators. For example, an admin group modification without related alarms can make it difficult to evaluate the incident’s severity. This is illustrated in Figure 1, which shows two alerts: one is a user being added to an admin group and another is an unrelated alert of malware execution.
This exemplifies why we need suspicious sign-ins to make sense of multistage compromises. In Figure 2, we can see an adversary made a suspicious connection to Host A where User 2 was added to an admin group. Subsequently that user was used to sign into Host B where another user was used to execute malware. Using alerts from suspicious sign-ins we can now correlate these two alerts.
To create suspicious sign-in events, we employ custom ML/AI models tailored to each client’s environment. These models analyse authentication behaviour using Auto-Encoders, learning the network’s typical sign-in patterns to identify anomalies with high accuracy. Importantly, each model is trained locally for each customer to ensure data privacy and specificity.
By integrating our self-developed Suspicious Sign-Ins for on-premises AD, we provide enriched context for incidents such as admin group modifications. This additional context helps us more accurately assess the severity of incidents by correlating them with suspicious activity detected for associated accounts.
Additionally, it can elevate the severity of alerts in a network. Typically, informational severity alerts are not reliable enough, by themselves, to alert on. With suspicious sign-ins we can be more confident in their reliability. The two alerts in Figure 3 are an example of that.
By introducing suspicious sign-ins we can enrich these alerts and correlate them in an incident, as in Figure 4.
Overcoming Data Limitations
While we used the same principals in creating suspicious sign-ins as for Cloud (Entra ID) by using Auto-Encoders to analyse an environment’s authentications, the data sources differed significantly. In the cloud sign-ins we could access more attributes including, Location, User agent, Device status, and more fine-grained details such as the browser version. These are not available in on-premises AD, where we only have rudimentary data points: Source IP, Target Host, Account and Workstation name.
To enrich the basic features we’ve added correlating metrics, like the number of target hosts accessed by an IP or Account and the time a user was active on a target host. The number of hosts accessed by an IP, for examples, models the action of an attacker scanning a network. This data adds a layer of depth to the authentication behaviour analysis with which it becomes easier to differentiate between normal user behaviour and potential malicious actions.
With these additions to the authentication dataset, the Auto-Encoder model can accurately pick out authentications of a Red Team trying to stay under the radar from thousands of authentications.
Defending against Advanced Threats
This capability is particularly critical in combating Advanced Persistent Threats (APTs) and nation-state-sponsored hackers. These adversaries use sophisticated tools and techniques that enable them to infiltrate networks undetected. They may exploit compromised credentials, zero-day vulnerabilities, or compromised suppliers to gain access. Once inside, they bypass traditional detection tools and deploy custom malware to evade discovery.
Our proactive detection strategy focuses on profiling user behaviour and establishing tripwires for adversarial lateral movement within the network. These behaviours are typically necessary for adversaries to reach high-value targets. However, it’s important to note that such behaviours may also be exhibited by legitimate users. Consequently, suspicious sign-ins cannot be treated as standalone security incidents but must be used as correlation events alongside other security indicators.
Partner with Northwave for Comprehensive Security
At Northwave, we specialise in building tailored solutions to strengthen your organisation’s cyber security defences. From leveraging advanced AI-driven models to detecting suspicious sign-ins in both Entra ID and on-premises Active Directory, we help you stay ahead of adversaries, including sophisticated APTs and nation-state actors.
Contact us today to learn how our innovative tools and expert insights can enhance your security posture and protect what matters most.
We are here for you