Minor update to the DGN Purge command

For those who have (quite understandably) not been following the story unfolding around this tool, here's a quick timeline so far. I've largely pieced this together from emails, as I don't typically tag blog updates with a date (something it seems I ought to consider starting).

  • 12/12/12 – In response to a request from our Product Support team, the original implementation was posted to this blog.
  • 03/28/13 – I posted an update to the original post with a minor fix to handle corrupted drawings a bit better (Update).
  • 06/21/13 – A tool based on the (updated) code was published as an official tool (Update 2).
  • 06/23/13 – Jimmy Bergmark identified an issue with the code, in that it purged information needed by compound linestyles even when in use. We removed the tool and I submitted an updated version of the code for review (Update 3).

The code has since been reviewed by the developers of the feature (thanks, Scott and Markus!), and a couple of suggestions have been made. Based on feedback – and confirmation of the specific compound linestyle components that contain direct references to other components – I've adjusted the code to look for two specific classes, "AcDbLSCompoundComponent" and "AcDbLSPointComponent" (rather than performing pattern-matching). I've also added a small section of code that follows the references on "AcDbLSSymbolComponent" objects and erases their associated anonymous blocks: this should remove (or at least reduce) the need to call a traditional PURGE after DGNPURGE has completed.

All in all these are minor changes – in addition to the more important fix of properly handling compound linestyle components – but worth publishing here for people to pick up.

The following C# code is now being rebuilt and tested by our QA team, and should form the basis of the republished tool (along with the "ReferenceFiler" implementation that has remained unchanged from the original post).

using System;

using System.Runtime.InteropServices;

using Autodesk.AutoCAD.ApplicationServices.Core;

using Autodesk.AutoCAD.DatabaseServices;

using Autodesk.AutoCAD.Runtime;

using System.Collections.ObjectModel;

 

namespace DgnPurger

{

  public class Commands

  {

    const string dgnLsDefName = "DGNLSDEF";

    const string dgnLsDictName = "ACAD_DGNLINESTYLECOMP";

 

    public struct ads_name

    {

      public IntPtr a;

      public IntPtr b;

    };

 

    [DllImport("acdb19.dll",

      CharSet = CharSet.Unicode,

      CallingConvention = CallingConvention.Cdecl,

      EntryPoint = "acdbHandEnt")]

    public static extern int acdbHandEnt(string h, ref ads_name n);

 

    [CommandMethod("DGNPURGE")]

    public void PurgeDgnLinetypes()

    {

      var doc =

        Application.DocumentManager.MdiActiveDocument;

      var db = doc.Database;

      var ed = doc.Editor;

 

      using (var tr = doc.TransactionManager.StartTransaction())

      {

        // Start by getting all the "complex" DGN linetypes

        // from the linetype table

 

        var linetypes = CollectComplexLinetypeIds(db, tr);

 

        // Store a count before we start removing the ones

        // that are referenced

 

        var ltcnt = linetypes.Count;

 

        // Remove any from the "to remove" list that need to be

        // kept (as they have references from objects other

        // than anonymous blocks)

 

        var ltsToKeep =

          PurgeLinetypesReferencedNotByAnonBlocks(db, tr, linetypes);

 

        // Now we collect the DGN stroke entries from the NOD

 

        var strokes = CollectStrokeIds(db, tr);

 

        // Store a count before we start removing the ones

        // that are referenced

 

        var strkcnt = strokes.Count;

 

        // Open up each of the "keeper" linetypes, and go through

        // their data, removing any NOD entries from the "to

        // remove" list that are referenced

 

        PurgeStrokesReferencedByLinetypes(tr, ltsToKeep, strokes);

 

        // Erase each of the NOD entries that are safe to remove

 

        int erasedStrokes = 0;

 

        foreach (ObjectId id in strokes)

        {

          try

          {

            var obj = tr.GetObject(id, OpenMode.ForWrite);

            obj.Erase();

            if (

              obj.GetRXClass().Name.Equals("AcDbLSSymbolComponent")

            )

            {

              EraseReferencedAnonBlocks(tr, obj);

            }

            erasedStrokes++;

          }

          catch (System.Exception ex)

          {

            ed.WriteMessage(

              "\nUnable to erase stroke ({0}): {1}",

              id.ObjectClass.Name,

              ex.Message

            );

          }

        }

 

        // And the same for the complex linetypes

 

        int erasedLinetypes = 0;

 

        foreach (ObjectId id in linetypes)

        {

          try

          {

            var obj = tr.GetObject(id, OpenMode.ForWrite);

            obj.Erase();

            erasedLinetypes++;

          }

          catch (System.Exception ex)

          {

            ed.WriteMessage(

              "\nUnable to erase linetype ({0}): {1}",

              id.ObjectClass.Name,

              ex.Message

            );

          }

        }

 

        // Remove the DGN stroke dictionary from the NOD if empty

 

        var nod =

          (DBDictionary)tr.GetObject(

            db.NamedObjectsDictionaryId, OpenMode.ForRead

          );

 

        ed.WriteMessage(

          "\nPurged {0} unreferenced complex linetype records" +

          " (of {1}).",

          erasedLinetypes, ltcnt

        );

 

        ed.WriteMessage(

          "\nPurged {0} unreferenced strokes (of {1}).",

          erasedStrokes, strkcnt

        );

 

        if (nod.Contains(dgnLsDictName))

        {

          var dgnLsDict =

            (DBDictionary)tr.GetObject(

              (ObjectId)nod[dgnLsDictName],

              OpenMode.ForRead

            );

 

          if (dgnLsDict.Count == 0)

          {

            dgnLsDict.UpgradeOpen();

            dgnLsDict.Erase();

 

            ed.WriteMessage(

              "\nRemoved the empty DGN linetype stroke dictionary."

            );

          }

        }

 

        tr.Commit();

      }

    }

 

    // Collect the complex DGN linetypes from the linetype table

 

    private static ObjectIdCollection CollectComplexLinetypeIds(

      Database db, Transaction tr

    )

    {

      var ids = new ObjectIdCollection();

 

      var lt =

        (LinetypeTable)tr.GetObject(

          db.LinetypeTableId, OpenMode.ForRead

        );

      foreach (var ltId in lt)

      {

        // Complex DGN linetypes have an extension dictionary

        // with a certain record inside

 

        var obj = tr.GetObject(ltId, OpenMode.ForRead);

        if (obj.ExtensionDictionary != ObjectId.Null)

        {

          var exd =

            (DBDictionary)tr.GetObject(

              obj.ExtensionDictionary, OpenMode.ForRead

            );

          if (exd.Contains(dgnLsDefName))

          {

            ids.Add(ltId);

          }

        }

      }

      return ids;

    }

 

    // Collect the DGN stroke entries from the NOD

 

    private static ObjectIdCollection CollectStrokeIds(

      Database db, Transaction tr

    )

    {

      var ids = new ObjectIdCollection();

 

      var nod =

        (DBDictionary)tr.GetObject(

          db.NamedObjectsDictionaryId, OpenMode.ForRead

        );

 

      // Strokes are stored in a particular dictionary

 

      if (nod.Contains(dgnLsDictName))

      {

        var dgnDict =

          (DBDictionary)tr.GetObject(

            (ObjectId)nod[dgnLsDictName],

            OpenMode.ForRead

          );

 

        foreach (var item in dgnDict)

        {

          ids.Add(item.Value);

        }

      }

 

      return ids;

    }

 

    // Remove the linetype IDs that have references from objects

    // other than anonymous blocks from the list passed in,

    // returning the ones removed in a separate list

 

    private static ObjectIdCollection

      PurgeLinetypesReferencedNotByAnonBlocks(

        Database db, Transaction tr, ObjectIdCollection ids

      )

    {

      var keepers = new ObjectIdCollection();

 

      // To determine the references from objects in the database,

      // we need to open every object. One reasonably efficient way

      // to do so is to loop through all handles in the possible

      // handle space for this drawing (starting with 1, ending with

      // the value of "HANDSEED") and open each object we can

 

      // Get the last handle in the db

 

      var handseed = db.Handseed;

 

      // Copy the handseed total into an efficient raw datatype

 

      var handseedTotal = handseed.Value;

 

      // Loop from 1 to the last handle (could be a big loop)

 

      var ename = new ads_name();

 

      for (long i = 1; i < handseedTotal; i++)

      {

        // Get a handle from the counter

 

        var handle = Convert.ToString(i, 16);

 

        // Get the entity name using acdbHandEnt()

 

        var res = acdbHandEnt(handle, ref ename);

 

        if (res != 5100) // RTNORM

          continue;

 

        // Convert the entity name to an ObjectId

 

        var id = new ObjectId(ename.a);

 

        // Open the object and check its linetype

 

        var obj = tr.GetObject(id, OpenMode.ForRead, true);

        var ent = obj as Entity;

        if (ent != null && !ent.IsErased)

        {

          if (ids.Contains(ent.LinetypeId))

          {

            // If the owner does not belong to an anonymous

            // block, then we take it seriously as a reference

 

            var owner =

              (BlockTableRecord)tr.GetObject(

                ent.OwnerId, OpenMode.ForRead

              );

            if (

              !owner.Name.StartsWith("*") ||

              owner.Name.ToUpper() == BlockTableRecord.ModelSpace ||

              owner.Name.ToUpper().StartsWith(

                BlockTableRecord.PaperSpace

              )

            )

            {

              // Move the linetype ID from the "to remove" list

              // to the "to keep" list

 

              ids.Remove(ent.LinetypeId);

              keepers.Add(ent.LinetypeId);

            }

          }

        }

      }

      return keepers;

    }

 

    // Remove the stroke objects that have references from

    // complex linetypes (or from other stroke objects, as we

    // recurse) from the list passed in

 

    private static void PurgeStrokesReferencedByLinetypes(

      Transaction tr,

      ObjectIdCollection tokeep,

      ObjectIdCollection nodtoremove

    )

    {

      foreach (ObjectId id in tokeep)

      {

        PurgeStrokesReferencedByObject(tr, nodtoremove, id);

      }

    }

 

    // Remove the stroke objects that have references from this

    // particular complex linetype or stroke object from the list

    // passed in

 

    private static void PurgeStrokesReferencedByObject(

      Transaction tr, ObjectIdCollection nodIds, ObjectId id

    )

    {

      var obj = tr.GetObject(id, OpenMode.ForRead);

      if (obj.ExtensionDictionary != ObjectId.Null)

      {

        // Get the extension dictionary

 

        var exd =

          (DBDictionary)tr.GetObject(

            obj.ExtensionDictionary, OpenMode.ForRead

          );

 

        // And the "DGN Linestyle Definition" object

 

        if (exd.Contains(dgnLsDefName))

        {

          var lsdef =

            tr.GetObject(

              exd.GetAt(dgnLsDefName), OpenMode.ForRead

            );

 

          // Use a DWG filer to extract the references

 

          var refFiler = new ReferenceFiler();

          lsdef.DwgOut(refFiler);

 

          // Loop through the references and remove any from the

          // list passed in

 

          foreach (ObjectId refid in refFiler.HardPointerIds)

          {

            if (nodIds.Contains(refid))

            {

              nodIds.Remove(refid);

            }

 

            // We need to recurse, as linetype strokes can reference

            // other linetype strokes

 

            PurgeStrokesReferencedByObject(tr, nodIds, refid);

          }

        }

      }

      else if (

        obj.GetRXClass().Name.Equals("AcDbLSCompoundComponent") ||

        obj.GetRXClass().Name.Equals("AcDbLSPointComponent")

      )

      {

        // We also need to consider compound components, which

        // don't use objects in their extension dictionaries to

        // manage references to strokes...

 

        // Use a DWG filer to extract the references from the

        // object itself

 

        var refFiler = new ReferenceFiler();

        obj.DwgOut(refFiler);

 

        // Loop through the references and remove any from the

        // list passed in

 

        foreach (ObjectId refid in refFiler.HardPointerIds)

        {

          if (nodIds.Contains(refid))

          {

            nodIds.Remove(refid);

          }

 

          // We need to recurse, as linetype strokes can reference

          // other linetype strokes

 

          PurgeStrokesReferencedByObject(tr, nodIds, refid);

        }

      }

    }

 

    // Erase the anonymous blocks referenced by an object

 

    private static void EraseReferencedAnonBlocks(

      Transaction tr, DBObject obj

    )

    {

      var refFiler = new ReferenceFiler();

      obj.DwgOut(refFiler);

 

      // Loop through the references and erase any

      // anonymous block definitions

      //

      foreach (ObjectId refid in refFiler.HardPointerIds)

      {

        BlockTableRecord btr =

          tr.GetObject(refid, OpenMode.ForRead) as BlockTableRecord;

        if (btr != null && btr.IsAnonymous)

        {

          btr.UpgradeOpen();

          btr.Erase();

        }

      }

    }

  }

}

If you'd really like to build this yourself in advance of the official tool being available, this comment has some brief instructions. Hopefully the official tool will be reposted soon, but this should be of some help to people with an urgent need who are unfamiliar with .NET development.

Update:

The AutoCAD DGN Hotfix is now available. See this post for more details.

31 responses to “Minor update to the DGN Purge command”

  1. Hi Kean.
    Would you modify the tool so that it can be processed by the equivalent figure ObjectDBX (of course for AutoCAD, not RealDWG)? I would like to give the Database object to the function, and the tool would have to process them in batch (a tool for loading drawings on my side).

  2. Hi Kean,

    Forgive my hasty inquiry; I've not tested this as I'm reading from my iPhone, but was curious to know if this tool does (perhaps will?) handle incremental blocks from DGN (FDOT's GeoPak in particular)?

    Cheers

  3. Kean Walmsley Avatar

    Interesting suggestion. I'll look into it.

    Kean

  4. Kean Walmsley Avatar

    Hi BlackBox,

    Quite honestly I don't know anything about those. This is very specifically for getting rid of unused data (imported from DGN linestyles) that cannot (currently) be purged using standard AutoCAD commands, even when not used inside the drawing.

    I wouldn't expect it to work beyond that, as it stands.

    Regards,

    Kean

  5. No worries; thanks for your time, Kean.

    Cheers

  6. I see that issued the official version of the tool. Would you convert it so it runs on the Database object fed in the argument function to make it possible to batch processing, as in the post above?

  7. Kean Walmsley Avatar

    As mentioned previously, I'll take a look. But I can't commit to when I'll get the chance.

    Kean

  8. When you deal with this problem, it will be useful a few particulars:
    As input are the Database and the Editor (null if batch processing mode). If it has been used in the current document, there is no problem. If the main function argument gave an external Database and Editor is null, there is an exception "eNotFromThisDocument" in the function:

    private static ObjectIdCollection
    PurgeLinetypesReferencedNotByAnonBlocks (
    Database db, Transaction tr, ObjectIdCollection ids
    )

    line 283 in the fragment:
    //Open the object and check its lineType.2point=2 point

    var obj = tr.GetObject (id, OpenMode. ForRead, true);

    Here pasted code: pastebin.com/yhrQVmL6

  9. Kean Walmsley Avatar

    Sure thing. I've dealt with these particulars and will post some code tomorrow.

    Kean

  10. Hi,
    If you are still looking for the batch process for this routine , I have published an Autodesk Apps based on the Code from this webpage.
    [Thanks to Kean for this code]
    @ Exchange App Store

  11. Kean,

    I originally sent a case to Autodesk earlier this year.. with a video if I remember correctly showing the behavior that copy pasting entities from DWG to DWG, where DWG 1 had some DGN entities impacted the second DWG. It was sent to dev team..so I am glad this has been resolved.

    We greatly need this tool as we have a huge DGN library.
    Is there any way I can keep it loaded without having to NETLOAD every time, as a plugin?

    Thanks

  12. Andre,

    Thanks for helping us track this down!

    Yes, you can absolutely get this loaded - either on startup or whenever the DGNPURGE command is executed - by making some Registry entries or modifying the app to make them itself when it loads:

    keanw.com/2006/09/automatic_loadi.html
    keanw.com/2009/05/creating-demand-loading-entries-automatically-for-your-autocad-application-using-net.html
    keanw.com/2010/03/creating-demand-loading-entries-automatically-for-your-autocad-application-using-c-f-or-vbnet.html

    You could also call NETLOAD from acad.lsp, if you prefer doing it that way.

    Regards,

    Kean

  13. Hi Kean,

    we've also struggled with the exception 'eNotFromThisDocument' in our batch processing tool.

    The following solved our problem:

    You've used the imported function acdbHandEnt(HANDLE, REF ENAME)
    for retreiving an ObjectId:

    "
    using (Transaction tr = db.TransactionManager.StartTransaction())
    {
    for (long i = 1; i < handseedTotal; i++)
    {
    // Get a handle from the counter
    var handle = Convert.ToString(i, 16);

    // Get the entity name using acdbHandEnt()
    var res = acdbHandEnt(handle, ref ename);

    if (res != 5100) // RTNORM
    continue;

    // Convert the entity name to an ObjectId
    var id = new ObjectId(ename.a);

    // Open the object and check its linetype
    var obj = tr.GetObject(id, OpenMode.ForRead, true);

    ...
    "

    Maxbe I'm wrong but I assume that the function call to acdbHandEnt() is checking the database of the current active document.
    Maybe this leads to the eNotFromThisDocument exception mentioned by badziewiak.

    This is what we did to get it working:

    "
    RXClass rxc_entity = RXClass.GetClass(typeof(Entity));
    using (Transaction tr = db.TransactionManager.StartTransaction())
    {
    for (long i = 1; i < handseedTotal; i++)
    {
    // --> create a new handle
    Handle h = new Handle(i);

    // --> try to get the object id from the passed database
    ObjectId id = ObjectId.Null;
    if (!db.TryGetObjectId(h, out id) && !id.Equals(ObjectId.Null))
    continue;

    // --> check for an entity
    if (!id.ObjectClass.Equals(rxc_entity))
    continue;

    // --> as we retrieved the object id from the passed database,
    // --> the following call doesn't raise the exception
    // Open the object and check its linetype
    varobj = tr.GetObject(id, OpenMode.ForRead, true);

    ...
    "

    Maybe I missed s.th. and you could point out the differences between the funtions acdbHandEnt() and db.TryGetObjectId() for that specific case.

    Thanks a lot for this.

    Chris

  14. Hi Chris,

    Yes, that was basically an oversight. See the comments in this follow-up post to see how it came about:

    keanw.com/2013/08/purging-dgn-linestyles-from-drawings-not-open-in-the-autocad-editor.html

    Ultimately they're comparable, I just overlooked its existence when I first created the sample. As a rule, it's a good idea to reduce the number of P/Invoked functions in your code, as it adds fragility between releases, in particular. So TryGetObjectId() is the way to go, overall.

    Regards,

    Kean

  15. Hi Kean,

    Just a thought, you use:
    for (long i = 1; i < handseedTotal; i++) ...
    to scan the whole database, it's a nice way i use too searching for proxies for example. But here you're looking only for entities to check their owners.
    Won't it be more efficient to only iterate all BlockTableRecords ?

  16. Hi Gilles,

    We could certainly save some cycles by doing that... my original intention (and it's been a while, so I forget) was to make sure we checked all relevant object references, to perform the purge as thoroughly as possible. But as the code is only checking references from Entities - which by definition are owned by BlockTableRecords - then this does seem a good target for optimization. Again - this is just dipping back into the code after a long time away, so hopefully I haven't missed something.

    I haven't seen much feedback on the usage of the tool for some time, but will certainly keep this (along with using TryGetObjectId() instead of acdbHandEnt()) in a list of potential future enhancements.

    Thanks for the suggestion!

    Kean

  17. (Although iterating BTRs would presumably remove the need for acdbHandEnt()/TryGetObjectId(), thinking about it...)

  18. I seem to be having trouble getting DGNPURGE to get rid of the strokes:

    "Purged 12 unreferenced complex linetype records (of 12).
    Purged 0 unreferenced strokes (of 5842)."

    Any idea of what may be the issue? Thank you

  19. Also says this for each stroke

    "Unable to erase stroke (AcDbZombieObject): eNotAllowedForThisProxy"

  20. I've seen this in one or two drawings, but wasn't able to determine why they became this way. Somehow the stroke information has been turned into proxy objects - our best guess is that it happened when the data was round-tripped via an external system.

    In case it helps, there are apps on the Exchange Store that can help with removing proxies from DWG files (search for "Zombie Killer" - I haven't tried it myself, though).

    Kean

  21. Thank you,

    You're right, I searched for Autodesk Zombie Killer and easily found the link apps.exchange.autodesk.com/acd/en/Detail/Index?id=appstore.exchange.autodesk.com%3azombiekiller_windows32and64%3aen&autostart=true

    For anyone wondering, I downloaded and installed the app (quick and easy, can create an Autodesk ID if you don't have one). Then back in AutoCAD, just find the KILLZOMBIES command, and then I ran the DGNPURGE again which would then be able to get rid of the 5842 unreferenced strokes that it could not previously.

    Thanks

  22. Hi Brock,

    I think I've finally worked out why this happens. Try setting your DEMANDLOAD system variable to 3 (setting it to 1 would also work). I suspect that it's currently set to 0, which stops the appropriate Object Enabler module from being loaded when you open a DWG with DGN linestyle data in it.

    I hope this helps,

    Kean

  23. In the end it turns out some installations are missing some Registry entries... more details here:

    keanw.com/2014/01/update-on-purging-dgn-linestyles-from-autocad-drawings-using-net.html

    Kean

    1. i'm not as confident making registry modifications and would like to purge thousands of linetypes and text styles that are in most of my drawings now. is there a simpler way to get rid of all these?

      1. You may not need to modify the Registry, only if you see this behaviour.

        Another option would be to take a look at AutoCAD 2015, to see how it handles such files (I admit to not having looked at that myself, as yet).

        Kean

  24. I've been able to use the DGNPURGE command as needed since the beginning of the year with (usually) no issues, but today I'm stumped. When I try to run in on a drawing we got in from another company, I get an error box with this message at the beginning:
    See the end of this message for details on invoking
    just-in-time (JIT) debugging instead of this dialog box.
    Then, way down at the bottom after a lot of crap I don't understand is this:
    ************** JIT Debugging **************
    To enable just-in-time (JIT) debugging, the .config file for this
    application or computer (machine.config) must have the
    jitDebugging value set in the system.windows.forms section.
    The application must also be compiled with debugging
    enabled.

    For example:

    <configuration>
    <system.windows.forms jitdebugging="true"/>
    </configuration>

    When JIT debugging is enabled, any unhandled exception
    will be sent to the JIT debugger registered on the computer
    rather than be handled by this dialog box.
    Any thoughts?

    1. Kean Walmsley Avatar

      Hi Lisa,

      I think that's just a standard part of the dialog displaying unhandled exceptions. It's possible the drawing is corrupted or that you're using an older version of the tool (I haven't seen this kind of issue reported with the latest shipping version).

      Another suggestion would be to try the standard PURGE capability in AutoCAD 2015, which now handles DGN linestyle data.

      Regards,

      Kean

  25. Hi there,

    I get a null object ID sometimes in the PurgeLinetypesReferencedNotByAnonBlock Method. The following if clause can fix this:
    [...]
    // If the owner does not belong to an anonymous
    // block, then we take it seriously as a reference
    //
    // Chek the OwnerId first!
    if (ent.OwnerId != ObjectId.Null)
    {
    var owner = (BlockTableRecord)tr.GetObject(ent.OwnerId, OpenMode.ForRead);
    [...]

    1. Thanks for sharing, max.

      So far I haven't heard of anyone else experiencing this issue, but it's great you've shared what worked for you.

      Kean

  26. R.K. McSwain Avatar

    Kean, I'm curious about this. One solution that seems to work for this problem is to do this:
    (dictremove (namedobjdict) "ACAD_DGNLINESTYLECOMP") ......and then purge the drawing.

    Can you comment on why this may or may not do just as good as the solutions found here?

    Ref: theswamp.org/in...

    Thanks.

    1. Kean Walmsley Avatar
      Kean Walmsley

      The approach shown on this blog is much more selective: it will only delete linestyles that aren't in use, and is therefore safer.

      Kean

Leave a Reply to SZimmerman Cancel reply

Your email address will not be published. Required fields are marked *