How to upgrade Data Import/Export Framework to AX2012 R2 CU7

Hi all,

Recently Microsoft released CU7 for Microsoft Dynamics AX2012 R2. During testing of the beta version I came across an error when installing CU7 on top of AX2012 R2 where the Data Import/Export Framework (DIEF) was installed as well. I reported this to Microsoft and learned that DIEF would be part of CU7. The objects were transferred to the syp layer.

The feedback was voided, so there would be no supported upgrade path. This post will describe a possible workaround for upgrading DIEF to CU7.

The installation guide for Cumulative Update 7 (link) now contains the next caution for the Data Import/Export Framework:

When you try to install without removing the model you can certainly expect this error:

The log file will give you the next details:

AxUtil call returned errors:
The model contains a customization of a configuration key. The model cannot be imported because you can overlayer a configuration key from the patching layer only.

So you are indeed forced to remove the DIEF model from your current instance. But what happens if you wait to synchronize the database until you installed CU7 completely? Well the old DIEF tables will be dropped because of changed ID’s.

Can you upgrade without losing settings, historical data and customizations?

This is the question I had and would like to have an answer on. As Microsoft does not support it, I explored other options. It is possible but read this post carefully and be aware that this post has no warranties, and confers no rights. When you follow the next steps be sure you first test it carefully in a separate environment (just like I did).

When upgrading an AX environment with additionally DIEF or DMF installed you will have to follow the next steps:

1. Export contents of all DMF tables
2. Create a backup of your modified/added DMF entities
3. Remove the Data Import Export Framework
4. Install Cumulative Update 7
5. Check DMF customizations
6. Import contents of DMF tables
7. Perform post update tasks

Step 1: Export contents

There are two options for exporting (and later importing) the data from the DMF tables.

[Option 1]

Exporting the contents of the DMF tables ironically can be done with the standard Data Export/Import existing in the System administration menu. This is because no foreign key relations were used within this tool. To create an Definition group can be time consuming if you want to include all tables. As I’m a bit lazy as well I created the next job for it:
/// </summary>
/// This job is used for creating a defintion group with relevant tables for DIEF upgrade 
/// </summary>
static void DMF_CreateExpImpGroup(Args _args)
{
    SysExpImpGroup      sysExpImpGroup;
    SysExpImpTable      sysExpImpTable;
    UtilIdElements      utilIdElements;
   
    #Define.GroupId(‘DIEF_upgr’)
    ;
 
    ttsBegin;
 
    if (!SysExpImpGroup::exist(#GroupId))
    {
        sysExpImpGroup.GroupId  = #GroupId;
        sysExpImpGroup.Name     = “Upgrade Data Import/Export Framework”;
        sysExpImpGroup.insert();
    }
   
    while select utilIdElements
    where utilIdElements.recordType == UtilElementType::Table
       && utilIdElements.name like ‘DMF*’
    {
        if (!sysExpImpTable::exist(#GroupId, utilIdElements.name))
        {
            sysExpImpTable.GroupId          = #GroupId;
            sysExpImpTable.ConvTableId      = utilIdElements.id;
            sysExpImpTable.convTableName    = utilIdElements.name;
            sysExpImpTable.insert();
        }
    }
 
    ttsCommit;
 
    info(“Job completed”);
}

This will result in the creation of a new Definition group and all tables starting with DMF will be attached. Now you can export the contents of the tables.

[Option 2]

The second option would be using the Test Data Transfer Tool (beta). This tool will be more suitable if the tables contain many records. This tool is much faster when you have already experience with it. Recently I blogged about this tool with a tip. (link)
This tip is about using an exclude syntax to include only a set of tables containing some similar characters. In this case you can create an exclude file with the next syntax:

Running the DP.exe with the correct parameters in a command prompt box will export the contents of the tables to a specified directory.

Step 2: Create a backup of modified/added entities

This step is for your peace of mind only or when you want to remove the customizations on the Data Import/Export Framework prior to upgrade to CU7 as well. You can choose for yourself to export a model or xpo file. Within my test I did not had to delete the newly created entities. I have to admit that I did not had changes on entities delivered out of the box in my test environment.

Step3: Remove Data Import Export Framework

The DIEF binaries shipped with CU7 do have updates as well and so it is needed to remove the binaries before installing CU7. The old binaries will not work correctly with CU7 and causes errors. You can use Add/Remove Programs, to remove these binaries on eventually several machines.

Also you have to remove the model from the application you want to update. See “How to: Remove (Uninstall) a Model” (link).

Step 4: Install Cumulative Update 7

To install the updates from CU7, you have to read the installation guide for Cumulative Update 7 carefully. It contains important information how to perform this update. E.g. CU 7 contains database schema changes that require that you reinitialize the model store. The guide will tell you when and how.

When starting up the AX client you will have to run the software update checklist. One part of this checklist is to merge code automatically. My findings on the merge will be described in the next step.

 Step 5: Check DIEF customizations

When you look into differences you probably have to merge or adapt some coding. This will depend on your changes. One thing to keep in mind is the Entity type enumeration. When you have created new entities in Data Import/Export Framework, it will also add an option for your new entity in this enum and increases the value automatically. As CU7 contains many new entities there are values used twice. In the next sample my new Exchange rates entity had the Enum value 73 which is now claimed by Budget.

So you have to increase the Enum values of your own created entities. It is wise to leave a gap for future Microsoft investments. E.g. you can use the value 200. When you later create new entities yourself, it will increase the highest value, so AX will use the value 201. When you create a new entity the first time it is also advisable to have a gap. One concern I do have is the limitation of 250 options for AX enumerations. Will this enumeration be usable if we need more than 250 entities?

Step 6: Import contents of DMF tables

When you have completed the software update checklist, you can import the data which was imported in step 1. Depending on the method you have chosen you need to import the .dat file or you have to use the test data transfer tool to import the data back in AX. When you use the definition groups, be sure to empty all DMF tables prior to import. In my test all data was imported correctly, but I did not use all entities in my test.

When you have imported the data, there are two problems which needs to be corrected. This is caused by the enumeration as explained in step 5 and due to a data model change. Also you may want to have the new entities setup in the target entities as well. So to solve these issues, you have to complete step 7 as well.

Step 7: Perform post update tasks

[task 1]

When importing the data your entity with enum values  73, 74 and perhaps more do now have another label. This should be corrected. This is not possible within the form so you would have to take other actions. Also about 80 new entities should be setup. As told earlier in this document I do not like to do things manually, so I created a job to do this for me as well. This job will set the entity type to ::Custom. You could change it to have per entity the correct value.

Also in CU7 there is a new field called “Application module”. The job below does not cater for finding the appropriate module.
/// <summary>
/// This job is used for creating different Entities setup based on entity type(i.e. DMFEntityType)
/// When an Entity already exists it will be skipped and the current settings will remain
///
/// Before creating it will also check if the enumeration of not default entities is correct.
/// The DMFEntityType will then set to ::Custom
/// </summary>
static void DMF_CreateTargetEntities(Args _args)
{
    int                 i;
    boolean             isFound;
    DictEnum            dictEnum;
    DMFEntity           dMFEntity;
    dMFEntity           dMFEntityCurrent;
 
    dictEnum = new DictEnum(enumNum(DMFEntityType));
 
    // Enumerate all DMFEntityType based on their enum value, skipping ::Custom
    ttsBegin;
    for (i = 0; i < dictEnum.values(); i++)
    {
        if (dictEnum.value2Label(i) != enum2str(DMFEntityType::Custom))
        {
            dMFEntity.EntityName = dictEnum.index2Label(i);
            dMFEntity.Type       = dictEnum.index2Value(i);
            isFound = dMFEntity.defaultEntity(dMFEntity.Type);
 
            //Check existing record for changed types
            while select forUpdate dMFEntityCurrent
            where dMFEntityCurrent.Type == dMFEntity.Type
            {
                if (dMFEntityCurrent && dMFEntityCurrent.TargetEntity != dMFEntity.TargetEntity)
                {
                    dMFEntityCurrent.Type   = DMFEntityType::Custom;
                    dMFEntityCurrent.update();
                }
            }
           
            //Create new if not exists
            if (dMFEntity.validateWrite() && isFound &&
                ! DMFEntity::find(dMFEntity.EntityName))
            {
                dMFEntity.insert();
            }
        }
    }
    ttsCommit;
   
    info(“Job completed”);
}

[task 2]

The form Processing group will not show any record until you fill a new table DMFDefinitionGroupDataArea. Per record in the DMFDefinitionGroup you will need a record in this new table. Entering a value in the Definition group field is sufficient. You can leave the field DataArea empty. When there are many processing groups created, you can perhaps the next job to do the hard work for you.

/// <summary>
/// This job is used for creating related records in the table DMFDefinitionGroupDataArea for upgrade
/// </summary>
static void DMF_CreateDefGroupDataArea(Args _args)
{
  DMFDefinitionGroup dmfDefinitionGroup;
  DMFDefinitionGroupDataArea dmfDefinitionGroupDataArea;
;
  ttsBegin;
  while select dmfDefinitionGroup
  {
      select firstOnly dmfDefinitionGroupDataArea
       where dmfDefinitionGroupDataArea.DefinitionGroup == dmfDefinitionGroup.DefinationGroupName;
       if (!dmfDefinitionGroupDataArea.RecId)
       {
           dmfDefinitionGroupDataArea.DefinitionGroup = dmfDefinitionGroup.DefinationGroupName;
           dmfDefinitionGroupDataArea.insert();
       }
  }
  ttsCommit;
  info(“Job completed”);
}

That’s all for now. Till next time!