You need Just-In-Time (JIT) reasonable access for all relevant parties working with your Azure assets! Read along here as Magnus Mårtensson, CPO in Devoteam M Cloud Denmark and Microsoft Regional Director, walks you through the complex and powerful world of Privileged Identity Management (PIM).

In this article I bring you along for a deep dive into the specifics of Azure Active Directory Privileged Identity Management (PIM) and why it matters more than you think. 

Giving the wrong levels of access to your Azure assets and service environments is probably the leading cause of Cloud headaches. It is responsible for all kinds of downstream trouble including, but not limited to:

• Environment drift – when anyone can modify any setting they presume should be modified. This leaves you with differing environments when you need them for testing. 
• Bad actor – when an employee is disgruntled, malicious, or maybe even is spying on your company.
• Human error – when someone for example accidentally picks the wrong database to run a change script on and were only able to do so because they had too much access in the first place. 
• Stolen credentials – via stealthy social engineering like phone or email scamming or stolen hardware.

What you need is Just-In-Time!

As professional people, we do not want to make mistakes. And the best way to avoid making mistakes is to limit the risk of making them. At the same time, you need enough access to do your job at exactly the moment you need to, or else efficiency suffers, and your workday becomes cumbersome. What you need is for employees to have Just-In-Time (JIT) access, in the form of “least privilege” access, to perform only the specific tasks they themselves need to do.

In most cases, JIT means that a person does not have very much access at all. That means: no access to privileged roles that provide high security access, such as modifying production environments, or accessing production customer data. When someone needs access to a privileged work task, however, it is possible to elevate a dormant role to perform work in the protected areas.

“Least privilege” is a principle that states a person should have access to what they need to do their job, but no more. Bothering employees with having too much access as permanent assignments will often lead to mistakes that can carry a very high cost. This is not only very unnerving for employees, but usually leads to dissatisfaction, high employee turnover, data breech, and other negative outcomes. While people must be enabled in a timely manner, and need to feel empowered to do their job, my experience is that they do not want to have much more access than needed.

Nobody wins when everyone has access…

An old customer of mine had hundreds of customer databases. One database per customer environment. Because the company did not have a setup that enabled JIT and least privilege, they granted too much access to too many employees. It was bad to the extent that they in fact granted permanent database-server level access to support staff, including new hires, to 200 or more databases! Keep in mind that an ordinary support case would only require temporary access to just one database. As you can imagine the security situation, the data protection compliance, and the surface area for making a rookie mistake was horrible. Situations like this are so dangerous and completely unacceptable in any modern company.

The correct and powerful tools to stay safe and sound in your company while being empowered to effectively do the job and success is paramount. Enter stage left: Privileged Identity Management in Azure Active Directory.

What is AAD PIM?

The Microsoft Azure Active Directory (AAD) is the identity and access service underpinning all of Azure and many other services from Microsoft. In fact, a while ago Microsoft’s new “Windows Azure” Platform did not yet support any Active Directory services. We used to sign into Azure with non-corporate Microsoft Accounts to access corporate resources, because it was the only option available. AD services in the Cloud emanated instead from the Microsoft Exchange evolution we know as Office 365. 

When the Cloud AD service for office workers had hundreds of millions of daily sign-ins, the same service was also added to and modernized Microsoft Azure. Today we have one cloud-based Active Directory service in the Cloud called AAD to govern access to all Microsoft services.

The Privileged Identity Management features (PIM) of AAD are part of the premium service offering, which means they require a “P2 license”, which is a higher cost per user. The obvious question is if it is worth the extra cost? The answer is indisputably – YES! 

Why PIM is well worth its cost

PIM allows you to set up access that is not permanently active, but instead access that a user is eligible to activate when needed. A worker can have the admin access they need – to perform high security maintenance – only when the role assigned to them is activated. There are many features in PIM that make this a dream to work with:

• Activation can require a logged justification from the user asking for access. 
• All PIM activation activity is traced and support audits. 
• Requests to elevate can have required manager review with the option to approve or deny the action. Even denied requests are of course audited. 
• You can even connect a work item ID to an elevation request. 
• When you activate an eligible role, it may be required to provide Multi Factor Authentication (MFA), for example using your mobile phone, to complete the request. This adds one additional level of security exactly where it is needed and prevents access with stolen account credentials.

Security when using PIM drastically increases, and user risk of making mistakes is drastically decreased.

The risks of moving with the front line

There are a few downsides with PIM to that you should be mindful about. One glaringly obvious one is that some of the features of PIM are still in preview in Azure, which means that they do not yet have a service-level agreement (SLA) from the supplier, Microsoft. Can you then use those features in your production areas? 

Many enterprises today have taken that step, but only you can assess if that is appropriate for you and your company. The positive spin on this is that the feature set of PIM is already rich and expanding, allowing you fine-grained control of user access management. Moving with the front line can generally be very beneficial and well worth the risks. 

The downside to the fine-grained control offered in PIM is that companies commonly find themselves challenged to make PIM work for them instead of becoming an administrative nightmare, or a roadblock for employees wanting their jobs. 

An automation silver lining?

But the automation story for PIM is still evolving and improving in this very moment. In fact, automating PIM configurations have moved from being an Azure concern only into becoming a broader Microsoft concern for all service offerings. 

This transition takes time, and in the meantime you as the end-user of PIM do not yet have a fully completed automation story. Downsides aside, AAD PIM is incredibly powerful and very well worth getting into and using to empower your organization. My recommendation right now for PIM is to proceed carefully yet resolutely and take advantage of PIM for your organization!

How to use PIM – a deep dive

Slightly more technically then, how does PIM work? Here is the high-level overview. First, an administrator with enough access, sets up PIM for a user. There is also an option to set up PIM for security groups and assigning many users as members to them that all get the same PIM configuration. I will leave that optimization for the next post. When a user has eligibility to activate a role, they need to make a request for activation. The request can be configured to require written justification upon activation, and it is time limited – automatically revoked. 

Additionally, which is a great feature mentioned above, it can be set up to require MFA to complete an activation. When the request is complete, and if approval is required, an email is sent to the responsible manager, who can review, approve, or deny the request. If the request is approved, the requester is notified of the approved access. The activation is ready to take place. The employee now has an approved activation request and can proceed to activate the role and access that comes with it. After work is completed, either the access expires on its own, or can be deactivated. This is particularly useful, because the reason for having permanent access is often that “we don’t have a process to manage revoking it”. All this activity is logged and can be audited.

Your guide to mastering PIM:

In essence the following challenges with PIM need to be overcome for you to successfully leverage the service:

• Learning how to configure PIM access effectively and correctly. Again, the group management options are particularly good, but the automation story is currently lacking. 
• Understanding the scenarios for PIM, where it makes a good difference, and how different configurations of the same can target different scenarios. Both examples are equally valid, supported by PIM, and quite different from each other: 
  • Access to Global Administrator Role – the highest AAD role – probably always needs to be justified, approved, and MFA secured. Each time this access is needed it must be for a specific reason, maybe even tied to a work order number. 
  • Managing user accounts – Create new users, assign them an O365 license, and assign SharePoint permissions is an administrative role that is also protected. This role, however, can possibly be activated without manager approval. It is still audited of course, and MFA can be required to protect employee data. Making a single activation to receive all three roles required, AAD User Administrator, License Administrator, and SharePoint User Administrator, just-in-time when needed, is certainly very beneficial.
• Adapting your organization to learning to live with just-in-time roles and being effective doing so. There can be initial resistance when the company tightens security – “am I not trusted” – and “I don’t like change”. If PIM is loosely configured, it might not be as secure as required, but if it is very tightly configured it may be seen as an obstacle to being effective. The organization needs to adapt here, and PIM configurations need to be tweaked until they suit you well!

Change is rarely comfortable

That last point warrants some additional consideration. It is a story old as time – Who moved my Cheese* – where we need to be reminded that people generally do not respond very well to change. It is in the best interest of any company not to be careless with permanent access to critical systems. 

As pointed out, all bad things that can happen due to too much access, including data theft and security breaches, will eventually happen. It is only a matter of time, and you cannot afford that to happen! You have to think of your business value weighed against the uncomfortability of change. Of course, business value wins. At the same time, we need to be mindful that these changes may be hard won. Changing how people work is sensitive.

Getting started with PIM

There are a few straight forward steps to take to get started toward safer and more sound access control with PIM.

• Start a small POC with a few users. 
• Build the business case motivation for the AAD P2 licenses. 
• Prepare the upskilling for employees along with the motivations why this change is.

Good luck and stay safe and sound in the cloud!

Additional resources:

Wikipedia:

Principle of least privilege

*Who moved my cheese?

Azure Active Directory Privileged Identity Management AAD PIM:

What is Privileged Identity Management?

Originally posted at: https://www.devoteam.com/expert-view/mvp-insights-safe-and-sound-in-the-cloud-with-privileged-identity-management/

Correctly granting access to your Microsoft Azure resources is a process filled with potential pitfalls. Magnus Mårtensson, Azure MVP from Devoteam M Cloud, tells you how to do it the right way.

A big challenge for any company that uses Microsoft Azure cloud services is to grant access to your cloud resources in Azure in an appropriate and safe manner. It adds to the challenge that the administrative owner of the Azure accounts may be a less technical person who is responding to requests from ‘the tech people’ asking to be “unblocked to do their job”. 

The immediate risk is that the admin then grants full access to Azure “to be done with it” and intends to let the tech department sort itself out! Already, too many people have the wrong type of access. The potential issues that can arise from such a situation includes, but are not limited to, wasteful Azure spend, mistakes that impact production (and ultimately your customers), and security or data breaches.

I am here to tell you that using Azure to grant access to Azure may not be as straightforward as you would think. Why not? Read on and I will lay out the challenge and how it might be mitigated for your company.

The dual nature of identity and access for Azure

If the question is “who has access to what in Azure?” the answer is split in two: In Azure, access to your resources is granted to users originating from your Azure Active Directory (AAD). Azure and AAD are two different services. Do you see a potential challenge here? 

The problem, of course, becomes the division between the two systems. For instance, you cannot ask AAD what a user has access to in Azure when that information is stored in Azure. Likewise, you cannot ask Azure who is behind an identity because that information is stored in AAD. As you probably know, the names of people who have access in Azure are shown in the Azure portal. Did you also know that these names have been fetched by Azure from the AAD?

Access to one thing does not mean access to all

I will, for context, give a few examples on specific access grants stored in Azure. Access could, for instance, be permission to read settings on an Azure Web App resource. Access could be to manage (or contribute to) a storage account resource. A side is that data about a resource and data inside the same resource are two different things in Azure. Access can often be granted to manage a data storage resource but not at the same time granted to the data inside it. One could also have access to the data plane only (the data content) of the same resource, but not to manage the resource itself. It is possible to have both types of access at the same time, but those can be different grants.

The last initial thing worth noting is that Azure Active Directory is the world’s leading provider of identity and access management. Naturally, we want to take advantage of this strength when managing access to your critical Azure resources!

Beware of granting ghosts and zombies access in your Azure!

I want to tell you a story to illustrate some of the key points of access management. Meet “Casper” and “Dave”. They work at your company where they are part of your Azure team.

Casper and Dave need access to Azure resources to do their job. For this reason, both Casper and Dave have been granted access to a resource group.

As happens sometimes, both Casper and Dave later leave the company. The AAD account for Casper’s identity is erased. Casper is now in theory entirely gone – and consequently he ghosts the AAD. Dave‘s account, however, is NOT erased but instead just suspended, so Dave can no longer sign in. 

In both cases Casper and Dave no longer have access to your Azure resources, which is good. What is not good is the state of access in Azure now that they have left. Complications arise immediately when a new manager comes onboard.

An access nightmare

Meet “Sarah”. Sarah is a new manager in your company, and she joins the Azure team. Sarah needs to understand who has access to various Azure resources. She looks in Azure at the resource group where Casper and Dave had access and she is immediately confused.

It seems like there is a ‘ghost’ haunting the Azure resource group! Some unknown entity seems to have lingering access. Even if Azure asks the AAD which identity it is, the response will be that it is unknown since Casper’s identity does not exist in the AAD anymore. 

Afraid of what this might mean, Sarah reaches out to a person who has access in the same group, Dave. Worry builds as Dave does not answer Sarah’s messages. As we know, Dave left but when you look in access management for the resource group, it looks like he is still roaming around in Azure (almost like a zombie). Like with Casper, Azure asks the AAD who has this specific access. And the response will be Dave – even though Dave’s account is locked, and Dave no longer works in the company.

Sarah is now very confused and does not know what to do. Can we help Sarah?

Let’s do better!

This weird little story is too close to reality for many companies I am called in to work with. The current approach does not work well and we instead need to try something different. I want to be very clear that the method I am discussing next is the method we have employed to manage access to IT systems with Active Directory for decades. All I propose is that we take the same care in Azure, which unfortunately we as an industry, currently do not.

Why is has it become so bad?

A legacy reason for poor access control in Azure is that before AAD was available for access control in Azure, the only option was to access Azure subscriptions and resources using a Microsoft Account from Microsoft. Access to a subscription was completely binary. Either you did not have access to all the resources in the subscription, or you had full access to all of them. Remnants of this legacy still exist today but should be avoided and removed if you ever see it. The right way is to use AAD users and corporate accounts.

A more likely reason for the degraded state affairs for access control in Azure is instead related to a general inexperience in Cloud. Many of us are learning Azure while working in it, and access control is the first pitfall we stumble into. After a while, when we begin to understand what access might and should be in Azure, we find it hard to start over. On occasion I have been asked to reset access for users of Azure. That often leads to interesting conversations about why it is necessary to remove access previously granted. An employee might ask: “Am I not trusted anymore?”. 

The real stance must be that we wish to unburden our co-workers with the responsibility of too much access. Restricting appropriate access where it has been too lax is not a punishment, but a gift. Most people I speak to do not want too much access, only the access they need to do their job, and only while they are performing a work task that requires access.

Granting and managing access to Azure at scale: How to do it correctly

We need to use the full potential of the AAD, which is its tools for managing users and groups of users. Then we use a naming convention and automation tools to ensure the right group in AAD has the right access in Azure. We want to only grant access in Azure to Active Directory Security Groups and never grant access directly to AAD identity objects, such as machine accounts or human users!

Returning to the story, Sarah has employed two new engineers, “Cecilia” and “Daisy”, and proceeds to grant them access to some company Azure resources. Sarah creates an AAD Security Group which she grants access in Azure. She makes Cecilia and Daisy members of that group. Now, the engineers have membership granted in AAD in a manageable way, and only groups have access to Azure. AAD knows which users’ identities are members in what security contexts. When that needs to change, this is managed in the AAD.

Technical or administrative skills?

As hinted at just above, while it is important to always grant appropriate access, just-in-time when needed, and with least possible privilege, it is imperative to also be empowered to do that with uniformity and in the correct way each time. We discussed how it is common that an admin setting up an Azure commitment for the company is not necessarily experienced in setting up appropriate access. 

But why is the technical administrator for an Azure setup not doing it, then? The administrators are often not accustomed to Azure yet, and even when they are, the right access at the right time is not always something they have/or take the time for. 

They may become an unwilling bottleneck in the process, slowing down progress, and blocking employees from accessing Azure when they rightfully need to. Is it better then to grant full access to some team members? No, it is worse!

Invest in access!

The right access to Azure is a fundament worth investing in, so that you can feel secure as a company and grant your employees the freedom to be accountable for only the things they need to work with, while not needing to be bothered with any undue access they never asked for!

Keeping a tight ship while cruising toward the cloud, while challenging and uncomfortable at first, will be greatly rewarded down the line!

Originally posted at: https://www.devoteam.com/expert-view/vanquishing-azure-access-ghosts-and-zombies/

I want to share with you my approach to setting appropriate access in Azure and delegate specific access control to team leaders. (Original post on LinkedIn)

When granting access to Azure for users I observe that appropriately tight control over access is brushed over and mismanaged. Often too much access, such as owner, is granted, and to way too many people. One longer-term consequence is that the team eventually spend time searching for weird bugs that “only happen in production”, just to find – much later and under pressure from dissatisfied users and screaming managers – that the real issue was that production and test was not configured the same way. Of course, automated deployments go a long way to mitigate environment drift, but the challenge with too much access, or rather inappropriate access, is that people with too much access can go willy-nilly and modify things they thought was right. This works well until it is not. Errors will always happen when humans are involved. Another issue with too much access is compliance. It is hardly compliant to allow the whole team into production environments with permanent all access.

Privileged Identity Management

This article is not about Privileged Identity Management (PIM), but I figured I would mention that as well since PIM is great technology. With PIM a user may be eligible to have a certain access, but it does not hold that privilege on a permanent basis. PIM may be configured to require managerial approval for activation. This is great because then someone with appropriate responsibility allows critical high privilege access and it is always time limited. If an account is compromised that has PIM configured for all high privilege access that account has much less value for the intruder. In relation to PIM, I advise my customers to enforce an audit after each access elevation to investigate the purpose of the required elevation and to document what was modified in the accessed environment.

The way I design appropriate and professional access to the Azure environments that belong to a team or a service is to limit scope and grant more access in development compared to production. Typically, any service hosted in Azure has multiple environments correlating to the stage it is in. The notion of “DTAP” – Development, Test, Acceptance and Production is very common. I grant the whole team read access on all environments – so that they can see all the resources they are responsible for. I only grant contributor access to development and test. For acceptance and production environments I only accept access via automated deployments using a Service Principal. Deployments must be automated for production. Acceptance is the final staging ground before production, so it is treated the same way. To acceptance and production, it is appropriate to configure PIM as discussed above to potentially allow humans to modify these environments only after appropriate access elevation.

Avoid subscription level access

I do not create a new Azure Subscription for each new service environment, because I think that is just a very big and chunky thing to deliver. It is more than what was asked for and therefore inappropriate to give. If a team receives their own subscription with full access, they tend to become careless with resources and wasteful practices ensue. The only thing you get in access control if you create more subscriptions is – more subscriptions. There is no inherent value at all for access and security. A definite downside to granting access to a whole subscription is that the access granted is not only to all resource groups and resources in the subscription at this time. Access is also granted to future resource groups that will be created. I find that the scope of resource group for access is more appropriate and relates more closely to the required access for a scenario. Subscription scope leads to the mistake of granting too much access in the future.

Azure Role Based Access Control

Role Based Access Control (RBAC) in Azure allows to assign access using a certain role to a certain scope. For example, the whole team may be readers in the service environment, but only Alice and Bob may be contributors. A scope can be a resource, a resource group or a subscription. In fact, a scope may also span multiple subscriptions. There are multiple challenges with assigning direct access to users in Azure. I will discuss them below while instead applying access via security groups.

Access using Azure AD Security Groups

When I grant access in Azure via RBAC I avoid granting it directly to humans or to service principals. Instead, I create Azure AD (AAD) security groups – called Access Groups (AG). To them I grant the access of a specific role in a specific environment. For example, foo_p_contributor_AG might be a security group in AAD for contributors on the production environment for a service called Foo. I find that putting something unique, the service name, as the prefix works well with searching in AAD. Search for that prefix and you find all AAD objects that relate to it. I proceed to grant this security group the access role of contributor in Azure for the resource group for the service foo. At this point you might think – well this is nothing new at all Magnus, it is the way access has been granted in IT forever. I agree, it is. It is just that for some reason I have yet to find an Azure project in the wild that adheres to this reasonable standard. Instead, what I see is that whole teams are owners of whole subscriptions. And mayhem ensues.

There is another excellent reason for using AG’s to grant access in Azure. Those AAD objects are security groups that are specific to the resource groups in Azure. They exist while their target resource groups exist and are only used for that context. If you grant RBAC to an AAD object in Azure and later that object is removed, the access granted remains in Azure. Azure can no longer look up in the AAD what the access was granted to. This creates an access ghost in Azure. Here you see I have granted reader access to the user Demo-Dave (he works in my office) and to a demo security group:

If I delete “temporary demo group” from my AAD the picture looks like this:

Access is granted in Azure to an object which is unknown in the AAD. I call these access ghosts.

Furthermore, if Demo-Dave leaves the company and his account is suspended access looks like this:

Yes, the same. I have visited many companies over the years where I have noted that “I probably need to talk to Dave because he has access to this service” only to find out that “Oh, Dave doesn’t work here anymore, he quit six months ago”. Azure still remembers Dave.

Azure AD Security Group Owners

Another thing that is quite nice when it comes to AAD groups is that they can have owners.

Group owners may add and remove members in the group and even add and remove other group owners. This feature creates the opportunity to effectively delegate to team leaders the authority to add and remove people to contribute to the services they manage. In Azure RBAC the same requires the owner role for the scope. The owner role allows the user to grant any role to anyone, not only selected roles. This is again inappropriate and ill advised. By setting up a security group owner for the Access Group you delegate only the ability to grant the same access to Azure that has been granted to the group. This is usually exactly appropriate for the context.

The path to improve your security posture and control in Azure

I urge everyone to invest in Azure access control and do the following:

• Add appropriately scoped access groups with roles to your service environments in Azure! Do this when you create the service environment – Reader and Contributor is a great start and it covers about 95% of your use cases. Other roles can be granted upon request and involves setting up another access group.
• Avoid granting access to subscription scope or higher! Instead, grant access to resource group scopes. Granting access to resource scope only is often too fine-grained. I prefer to segment applications over multiple resource groups with related scopes such as foo_web and foo_db.
• Remove direct human and service principal RBAC assignments from Azure! Instead, make the same members of the appropriate access groups in AAD.
• Treat your service environments in Azure as the resource group AND the collaborating AAD access groups! Remove the access groups if the Azure resource is removed.

Following this approach, you are far less likely to ever hunt environment drift bugs, there will not be any ghosts in your RBAC assignments and there will also not be any access granted to people who no longer works in the company. When you get into delegating access control via security group owners you will discover how great it is and what a wonderful yet appropriately scoped empowerment it is.