Powershell Tip: FIM/MIM Explicit Disconnectors be gone… quickly!!!

Happy New Year!!! Hope everyone had a great 2017 and hopefully an even better 2018

If you know disconnectors then you definitely know how irritating the FIM/MIM GUI can be to convert a disconnector from explicit to normal or vice versa when you have potentially 10’s or 100’s of them… Joiner tab isn’t really friendly to select multiple disconnectors and “batch” convert them.

Thanks to Lithnet MIIS Powershell you can do the conversion of 100’s of them in 1 line and in couple of seconds

If you have a MA with 100’s of explicit disconnectors just run the following command (replacing the <MA NAME> with the ….. MA Name )

Done… Simples… You can ofcourse do the opposite as well if you wish i.e. Normal disconnectors -> Explicit Disconnectors.

KISS for MIM – Autosync

This one has been long overdue and about time that the community realizes how easy it can make your daily life with FIM/MIM.

A wonderful tool which keeps our engine purring like a kitten and all the cogs moving – Autosync written by @RyanLNewington

Since the past two year or so, it has evolved and now it is becoming mainstream with him making it more user friendly, GUI enabled and with KISS strategy (Love the strategy – Keep It Simple Stupid!!!).. It has now culminated as AutoSync.

We’ve worked together and I’ve done a lot of testing (somehow I have the notorious name of bug finder – I find them or say they find me in every product I’ve touched – FIM / MIM / Google / Lithnet / Oracle MA / Generic LDAP MA <– don’t even get me started with that one)

It’s extremely simple to use and self explanatory. It installs as a service, and has a easy to read XML config (no need to dive into as GUI makes it easy) and an “Execution Monitor” to show you the status of all  your runs.

Some of the features to list

  • Auto detect all MA in the environment and list them for configuration.
  • Automatically detect and prepopulate run profiles tab with best matching one.
  • Advance trigger capability to detect changes in the MA source and run only when needed
  • Ability to make changes to individual MA and restart only that component.
  • Ability to mention a list of MA’s not to run if a particular MA is running (to give priority or avoid deadlock situations)
  • One-Click button to create MPR / Set and service account in FIMService which helps in change detection in FIM/MIM Service.
  • Easy to configure and send mail notification of errors and customize it according to the errors you want to see / or ignore
  • Clear run history and store it as a maintenance task.
  • Different execution modes according to your choice (Supported / Exclusive / Unsupported)

The above is definitely not an exhaustive list and there are many more hidden and advance features which I can cover if it makes it to the final cut.

Let’s take an example on how we can utilize it in real world very easily. No coding experience needed.

We have an environment with 9 Management Agents (not the biggest in the world out there by far). We have some business rules which bind us to set provisioning times and deprovisioning times. Which means we need to keep the wheel moving – always and in sync.

Lithnet Autosync has a very easy to understand GUI interface. We can set which profiles to run to confirm an export (Confirming Import) or say which profile to run if the MA hasn’t run for X amount of time – scheduled import (which you can set as well).

For AD / LDS / LDAP MA we have change detection capabilities which basically runs the delta import / Full Import (whatever you configure) only when it detects a change for the system.

In the above example you can see I have also configured a “scheduled interval” which runs a full import for the MA every day at 2am. We do this to make sure the system is clean and up-to-date with the source. I have configured it for all the MA to run daily at different time intervals during non business hours so that the system is “clean” when we come back the next day. You can also configure them run exclusively to each other so that no clash occur (if that is a concern in your system).

You can also do change detection by writing power shell scripts and then running import run profile. We use an SQL based MA (ACMA) where we use such and example where the script looks at the Object Delta table and makes runs the DI (Delta Import) profile only if a change is detected. It can be as little as every second check but I run it every 5 second.

There are more such examples in “C:\Program Files\Lithnet\AutoSync\Examples” folder once you install it. We also have scripts to detect changes in a PostGres DB.

There is so much more we do with it. There are some MA which we use just for versioning purpose. So we simply have them as “unconfigured” so they are not used in the run.

Two most useful features I would like to show you are

Execution Mode

The text below are self explanatory but in a nutshell

  • Supported (Recommended): What Microsoft dictates – Never run sync profiles while any other profile is running. Import and export profiles of different MA can still run in parallel. When a sync profile runs its take precedence lock and doesn’t allow other profiles to run (they queue up)
  • Exclusive Mode (I call it the SloMo Mode): Every run profile in every MA are run exclusively. Slowest mode and possibly the safest mode if you are not concerned with timings and speed.
  • Unsupported Mode (Against the Bible): I use this quite often as I am confident of our environment and what we need in terms of SLA to provision / deprovision users. The difference in this mode v/s supported is that in this mode a sync profile can run while other MA are running their import / export profile.

Execution Monitor

This is where you can keep an eye on things like what is MA is running, what profile is running, what has the lock and what others are waiting to run next.

You can see a lot of live and useful information there.

  • You can see two MA are executing Full Import (FI) – A Play button indicates that
  • You can see that an MA has a lock and its preparing to start a Delta Sync (DS) and then after that it has Delta Import (DI) and EALL (Export All) queued up.
  • Another MA wants to run a delta sync (DS) and is waiting to take lock from the previous MA.
  • You can click on the MA and see the run history and logs of what it has done (depicted in the windows below)
  • As you would have guessed – my system is running in unsupported mode currently.
  • You can stop all execution and/or stop and cancel all runs from the button on top.
  • You can also uncheck the “Automatically start executors when service starts” so that when Autosync service starts it will not start any executions – handy when you want to check all config before you kick off the syncs.

There is so much you can do with the package that I haven’t even begin to list them here.

Hope this small post has given you some insight on the capabilities of Autosync and the potential it has to radicalize and make your environment more friendly and easy to use.

Let me know if you need a solution with AutoSync and I can probably help you with that….

Migrating Lost Identities from AD to MIM

Setting up a new identity system from scratch is so easy.. but we know that kinda never happens… we are always migrating from an old system to new or changing an existing system due to business changes.

We went through the same process whereby we went from an OpenLDAP (ODSEE) being our primary source to FIM/MIM pushing out to AD. We were syncing identities manually from ODSEE to AD via Perl scripts in the old days..

So after the original migration done we have a lot of identities in AD not joined to MIM i.e. Disconnectors as we know them. This was because as per our policy we always added / modified from ODSEE -> AD but never a delete. Thus it left a lot of users and groups in AD which never came across in the initial migration.

So once the migration was complete, the big mammoth task came to migrate the unknowns from AD -> MIM and then join the identities in AD.

Using few wonderful tools suite called Lithnet written by @RyanLNewington

NOTE: Always remember, my script are as-is and have lots of rules and logics specific to my environment and business needs. Its there to give a basic understanding on what can be done. You will obviously have to modify it for your environment and needs.

REQUIREMENTS

  • Migrate AD Disconnectors to MIM Service
  • Join them to AD
  • Make sure no existing values are overwritten in AD like sAMAccountName, DN, DisplayName etc
  • Make sure membership (member) is not lost. memberOf is not cared for atm as that will take care of itself (Point 4 below).
  • Sync Engine SHOULD NOT be provisioning. Stop the syncs (not service itself)

LOGIC

I wrote two different scripts – one to migrate users and one to migrate groups. There are some things to note here

  1. We have our own business rules engine called ACMA to which we provision all our identities which then generates all the required attribute values according to our business rules. So you will see each object being written to both FIMService and ACMA DB (SQL). Do have a look at it.. Its powerful yet so simple… Must have…
  2.  We have a unique ID which is pushed to all system called monashObjectID which is basically a random guid which is generated in ACMA DB. We use this to have a constant ID between all system and thus later helpful to join ID etc.
  3. Due to the above point, my script assumes that all objects in AD , if they have monashObjectID means they are connected to FIM/MIM. Thus, on reverse, if they don’t have it means they have to be migrated.
  4. Simplification of OUs: Due to our previous messy environment we have over 100-200 OU’s in AD where all these objects exist. In the new age, we have simplified it and MIM only manages 8-10 OU’s. Therefore according to the “Type” of the object being migrated they are moved to the respective new OU in AD prior to migration (you need rights to do so in AD if you migration account doesn’t so already). Again you can comment out the whole function if its not required.
  5. Groups gets tricky. As they can be member of another group and if they group doesn’t already exist in MIM, it will loose membership once imported, I decided to get around it by scripting a check that if the group has members and if all of those members have monashObjectID then dump them in a CSV file. Thus this migration is run multiple times.
    1. Create 1st batch of CSV for those groups who don’t have members or which all members exist in MIM (by checking their monashObjectID as that comes from MIM)
    2. Migrate them
    3. Export to AD – this will write out monashObjectID for the newly migrated groups
    4. Repeat step a-c till all are done.

USER MIGRATION SCRIPT

Will start will user migrations as they can be members of a group. Again remember, loads of logic for our environment.

CSV Requirements

The CSV requires the following fields

  • Name: sAMAccountName of the user to be migrated
  • Type: We have accounttypes in our environment like person / service / custom etc. Need to mention that (or modify code to remove it)
  • OrgUnit: We need to mention an OrgUnit DisplayName for each user to which it belongs to (our environment specific again)
  • Owners: This is to mention the owner / supervisor of the user. They are sAMAccountName of already existing users in MIM. They can be multiple with ; separating them.
Script

Rundown
  • Script records everything via transcript command so you can set and forget, come back later and see what broke or potentially could break (loss of membership my main concern). Saves it to D:\ADUserMigration-{CSVFileName}.log
  • Asks for CSV file on run.
  • If the  “Type” is missing in CSV it will skip that user (MUST in our logic)
  • Moves the account to respective OU in AD as per the account type.
  • Checks if the user doesn’t have monashObjectID – thus good to migrate.
  • Creates the object in FIMService using LithnetRMA
    • Depending on the type either we set expiry date to never (9999-12-31) or two years from date of migration
    • Checks the owner if its a person or a group and then looks it up if it exits in MIM and adds it as owner.
    • If no owner then we set a default account we have as owner
    • Checks OrgUnit and if found it adds it to the object else sets a default OrgUnit.
  • Checks if the memberOf exists in MIM and adds the user to those groups in MIM.
  • Creates the object in ACMA
  • Repeats the steps above for all users in the CSV
  • Sync the identities in MIM via MIIS Powershell
    • Disables MRE Provisioning
    • DI the FIMService
    • DS / Commit just the ones which were migrated into MV (as its prod others could come in from Portal which needs MRE rules and thus excluding them)
    • DI ACMA MA
    • DS / Commit just the ones which were migrated into MV – this should join them due to our join rules looking for same fimserviceobjectID which was written to ACMA DB during create
    • DI AD MA – all moved objects will now show in correct OU in CS
    • DS / Commit just the ones which were migrated into MV – this will join due to our join rules looking for same sAMAccountName.
    • Enable provisioning
    • DS all of them again to make sure MRE rules apply to those objects for any provisioning if required.

Done..

Speed

Can’t remember to be honest but was quite quick (had nearly none to do for accounts)

GROUP MIGRATION SCRIPT

Remember, loads of logic for our environment.

CSV Requirements

The CSV requires the following fields

  • Name: sAMAccountName of the group to be migrated
  • Type: We have account types in our environment like UserGroup / ServiceManagedGroup / IdMAuthZGroup etc. Need to mention that (or modify code to remove it)
  • ServiceID: For our environment – mandatory.
  • Owners: This is to mention the owner / supervisor of the user. They are sAMAccountName of already existing users in MIM. They can be multiple with ; separating them.
  • OrgUnit: We need to mention an OrgUnit DisplayName for each user to which it belongs to (our environment specific again.

Script

Rundown
  • Script records everything via transcript command so you can set and forget, come back later and see what broke or potentially could break (loss of membership my main concern). Saves it to D:\ADGroupMigration-{CSVFileName}.log
  • Asks for CSV file on run.
  • If the  “serviceID” is missing in CSV for a ServiceManagedGroup it will skip that user (MUST in our logic)
  • Moves the group to respective OU in AD as per the account type.
  • Checks if the group doesn’t have monashObjectID – thus good to migrate.
  • Creates the object in FIMService using LithnetRMA
    • Depending on the type either we set expiry date to never (9999-12-31) or two years from date of migration
    • Sets all as Security and Global groups.
    • Checks the owner if its a person or a group and then looks it up if it exits in MIM and adds it as owner.
    • If no owner then we set a default account we have as owner
    • Checks OrgUnit and if found it adds it to the object else sets a default OrgUnit.
    • Checks each member of the group and if they exist then adds them – else will spit out a warning that they will loose membership in AD (good to have logs eh?)
  • Checks if the memberOf exists in MIM and adds the user to those groups in MIM.
  • Creates the object in ACMA
  • Repeats the steps above for all users in the CSV
  • Sync the identities in MIM via MIIS Powershell
    • Disables MRE Provisioning
    • DI the FIMService
    • DS / Commit just the ones which were migrated into MV (as its prod others could come in from Portal which needs MRE rules and thus excluding them)
    • DI ACMA MA
    • DS / Commit just the ones which were migrated into MV – this should join them due to our join rules looking for same fimserviceobjectID which was written to ACMA DB during create
    • DI AD MA – all moved objects will now show in correct OU in CS
    • DS / Commit just the ones which were migrated into MV – this will join due to our join rules looking for same sAMAccountName.
    • Enable provisioning
    • DS all of them again to make sure MRE rules apply to those objects for any provisioning if required.

Done..

Speed

This has varied for me. It really depends on number of members for the group.. about 3-5 groups / min .. I did about 1500 groups with 90% of them having 0-10 members but the last few having upto 1100 members in 6.5 hrs.

 

Hopefully this will help someone and see what can be done by  combination of various tools out there.

Composite write back to FIMService via Powershell

So if you have a large number of objects to update in FIMService it could be complicated..

Lithnet FIM/MIM Powershell Module makes it so much simpler..

Below is a script I used to say update all accounts which have “accountType” = oldaccounts and set “accountBlocked” to true (my custom schema).

Some explanations

  • I have set PageSize = 100 which means if you are returning more than that number process only 100 at a time..
  • I have set “-AttributesToGet” which gives me only the attribute what I want

See how easy is that? You can do multiple operations before save-resource and thus do a bunch of changes to say 1000 users and in 10 saves it will be done!!!

PS: As you can see its simple for demonstration and not doing any error handling etc..

Some Great Addons for Admins in FIM / MIM Space

So if you have been playing with Microsoft FIM (Forefront Identity Manager) / MIM (Microsoft Identity Manager) you would already happen to know its not the best user experience out of the box if you have a complex environment.

Meet LithNet: https://github.com/lithnet

A lot of tools written by MVP Ryan Newington under whom I have had the privilege to work with. Have helped with lot of testing thus most of the codes following in the blog will contain cmdlets from the above.

Some of the MOST helpful tools in the belt are (which I use pretty much everyday)

There are some other great tools there like UMARE, ACMA, GoogleApps Management Agent which includes powershell commands to manage googleapps and some cool new stuff coming which I am not able to talk about atm 😉

If you need any explanation on how to use them, do hit me up here or the developer.

Follow the dev @Twitter