Sunday, 25 August 2013

VS2012 Quick Review

I recently had quite a few donations for HexEdit so I could afford to splash out and buy Visual Studio 2012 (aka VS2012 or VS11). I fully expected indispensable features to be removed or poorly reimplemented as I had experienced this time and time again when upgrading VS (see the list below of some major things). However, I had heard good reviews and was living in hope.


Good Things

I will start with the good things. First, VS2012 does not crash or run really slowly -- problems I have had after upgrading in the past. (But note that I am running Update 2.) It does seem to have a plethora of useful features, of which the ones I have tried worked well.

Many people have commented about the horrible colour scheme but, though I prefer the colours from any of the previous versions of VS, it's not a big deal. However, the new toolbar buttons (almost colourless) do make it hard to find the right command among the many toolbars/menus.

Now to the bad things...


Customization

One thing I noticed straight away is the painful toolbar/menu customization. Admittedly, this is the same as VS2010, but it is an absolute pain and I was hoping it had been improved.

The first thing I always do when I install a new version of VS, or install on a different system (and I use multiple versions of VS at different sites and on many different systems/VMWare images) is to create a toolbar of about 30 buttons that I commonly use then copy+drag them to the menu bar. (I create the toolbar first, as a backup, in case the menubar config stuffs up as it is prone to.) Then I can close all toolbars and have more room for code (and the multitude of VS tool windows). I then move all the tool windows around to where I want, as the default docking positions are awful and do not make use of multiple monitors.

This process takes 5 to 10 minutes in earlier versions (back as far as VC++4.X) by simply dragging (or copy+dragging) buttons from the Customize dialog or other toolbars/menus. In VS2012 it can take an hour or more to get things set up correctly. So it now takes at least 10 times longer than it took in 1995 - that is not progress!!

Possibly to compensate for this problem, VS now allows you to save your customizations (see Tools/Import and Export Settings). That way if your customizations get lost (as they do) or you want to install VS on a different system then you don't have to go through the same tedious process all over again. However, the settings export facility has some nasty surprises too (see below).

Last week, I bit the bullet and used the VS 2012 Customize dialog to put about 30 items on the menu bar. I then positioned all major tool windows (both in build and debug modes). Then I saved my settings (Menu and Command Bar Customizations + Windows Layouts). I probably spent about 2 hours on this, getting everything just right.

A few days later I somehow managed to open two copies of VS2012, which lost all my customizations! Well at least I had my settings backed up. But when I tried to restore my saved settings I got this message:
Your settings were imported, but there are some warnings. 
Warning 1: Category 'Window Layouts' ({eb4ba109-a9db-4445-bd09-e7604bcdce8A}) could not be migrated because the author of the category did not provide support for migration.
None of my customizations were restored!!! I have now gone back to using VS2008, as I don't feel inclined to repeat the above dispiriting procedure.

In summary the customization problems, when compared to earlier versions of VS2012, are:
  • menu/toolbar customization is slow and tedious compared to simply dragging
  • are given no information on the commands (apart from their name)
  • the lists in the Customization dialog suffer from the same horrible UI design bug as the Bookmarks window of not having an "append" slot (see the VS Bookmarks section in my blog post: Asymmetric Bounds)
  • it's generally confusing and counter-intuitive compared to old system
  • no way to backup menus/toolbars by dragging them onto a different toolbar

Keystroke Macros

Every editor I have used in almost 30 years has provided some sort of facility to record and playback keystrokes. This included PMate from Phoenix Technologies (1984), Brief (1985 - 1988), Emacs (1988 - 1991), Epsilon (1991 - 1996), VS IDE editor (1996 - present). So it was with great surprise that I discovered that VS2012 did not have this ability.

I normally map Record and Playback to F7 and F8, as I use them so often. When I tried to do this in VS2012 I spent ages looking for the commands to map. To me it was unfathomable that they had left this out so I just kept looking. Eventually a Google search revealed that many others had discovered the same horrible truth.

One justification for the omission (from someone at Microsoft) was that less than 1% of people use it. The problem with this logic is:
  • to the 1% who do use it, it's essential
  • for those who don't, ways should be found to encourage them to make use of such a time-saving facility
  • I could find dozens of features that are used by less than 0.1% - why not drop those instead?
  • I actually find it very unlikely that 1% is anywhere near correct

Previous Upgrade Disasters

Visual Studio has a bad history of removing or castrating good features when moving to a new release. (MS seems to be obsessed with rewriting it every few years.) Here are a few examples.
  1. In VC++5 the instant help (ie, the context help you get when you press F1 on a keyword or function name) was upgraded to use HTML help files. In 4.X you could get instantaneous help on a C standard library function, etc. In VC++5 this now took a very noticeable and distracting amount of time (up to 30 seconds from memory) and sometimes failed altogether.
     
  2. In VS.Net (aka VS2002) MS completely rewrote the IDE which meant that a number of features simply disappeared. One example is the named bookmarks that I found absolutely invaluable in keeping track of different areas of code in large legacy C projects. In the new VS, bookmarks could not even be given a name and were only remembered if they were in files that were open when you closed the project. Since then bookmarks have come back even better but why does VS have to take 2 steps backward before taking 3 (very slow) steps forward.
     
  3. I do a lot of searching (I press Ctrl+F3 on average once per minute!) and like everyone I often have a search fail for no apparent reason. This was why i really loved the search toolbar buttons that allowed you to toggle different search options (whole word, case-insensitive, regular expressions). In VC++6 and earlier I always added these buttons to the menu bar so that I could immediately see if they were on or off. This makes it so easy to check your search options and work out why your search failed.

    These toolbar buttons failed to make it into VS.Net. Though they have made a re-appearance in VS2012 in the little search tool that appears in the top right corner of the IDE editor window that appears when you press Ctrl+F.
     
  4. The very useful XML editor disppeared in VS2008.

Conclusion

I have gone back to using VS2008 which is a shame as there are a lot of things to like about VS2012. I still sometimes use VS2012 for things like the analysers or for testing C++11 language features (like lambda functions).

I would prefer to do all my development in one environment but until toolbar/menu customization is made easier and/or settings can be easily and reliably saved and restored I am not confident using the VS2012 IDE. I also sometimes need keystroke macros so would need to use a different editor for that.

VS2012 also needs to provide an alternative set of toolbar icons with a bit more colour so that different commands are more easily distinguished and recalled.

Tuesday, 6 November 2012

Passwords

I received an email some time ago asking whether I could add a facility to the Hex Edit Password dialog (see Operations -> Encryption -> Password...) to check the quality of the password entered.

I have seen web sites that check for various rules and I don't believe they are useful (except for a minimum length check as already provided in HexEdit).  I will explain this and how to create good passwords.  If you haven't the time to read the whole post, at least check out the summary at the end.

Password Validation Rules

First, a good password is not just one that is hard to break; it is important that it is easy to remember and type.  Second, whatever rules the password checker imposes affects the passwords that are chosen, which can itself create an avenue of attack.  Third, most of the time these rules don't ensure good passwords anyway.

I used to use a system for creating passwords and it was always really annoying when I tried a very good password but the web site prohibited it because of some rule, like it does not contain an uppercase letter or a digit.

Good passwords can fail these tests.  Moreover bad passwords can pass them.  You may have heard the story of one O'Reilly who worked in a high security (DOD) environment and for many years just used these passwords: "O'Reilly1", "O'Reilly2", ...  These passed all the rules (mixed case, punctuation, digits, etc) but were very definitely not good passwords.

The best approach is, as always, education.  The password creator should be aware of how passwords can be broken and what are good and bad password practices.

What Are Good Passwords?

There are several things to consider.  First and foremost the password must be hard to crack.  By far the best way to ensure this is to make sure the password is long enough.  I personally use passwords of at least 30 characters.  This may seem like too much typing, but it is often easier to type several words than to type a short jumble of random characters.

You probably know the commonly cited guidelines for passwords: that they should not be a word or name (in any language), involve your personal details such as car registration number, dog's name, SSN, etc.

What you are not told is that passwords should be easy to remember.  Passwords that are not easy to remember get written down and left in convenient places.

Another good practice is to avoid typing your password while anyone is watching, especially if they are holding a camera (or phone).  If you suspect that someone saw you type it and was trying to memorize it then you should change it immediately.

Difficult Passwords

I am currently doing some work contracting for a large company.  They have a lab of dozens of test systems and someone keeps creating passwords for these systems using an English word then mixing things up by making some letters uppercase and using Leet codes (eg E => 3, I => !, etc).  Here is an example:

  PaTh3t!c

There are so many things wrong with this approach.  When I am given such a password it can often take me many times to type it in correctly even when reading it.  I have seen other people just give up thinking they have been given the wrong characters.

Not only is it hard to type but it is also hard to remember.  After a while you remember how to physically type the password but forget the actual characters.  This can be a problem if you need to write the password down for another person (and you have to write these types of passwords down to have any chance of getting them right).

That's the other problem with this sort of password - it's so hard to remember that people have to write it down.  I am always finding little bits of paper with passwords written on them.  That's not good security at all.

Enforcement

I was a UNIX system administrator for many years and the one thing I learnt about passwords is that the worst approach is to try to force people to use good ones.

One policy that most operating systems provide is to force users to change their password regularly.  I really don't see the point in changing passwords more often than every year or so, as long as the password is a good one and you do not suspect that it has been compromised (in which case you should change it straight away).

Forcing people to think of a new password every few months, or less, means they will not try to think of a very good one.  Many Windows servers (ie, domain controllers) force monthly changes - I suspect this is some sort of default that someone at Microsoft thought was a good idea.  In this situation people invariably use the same password with slight variations.  An informal survey I conducted in an office with this policy showed that 100% of people just used the same password but added an incrementing number to the end of it.  Using this strategy some had reached large numbers (ie well over one hundred) meaning they effectively had been using the same password for more than a decade.

What I Do

I have used the same method of creating passwords since I was a UNIX system administrator and became aware of their importance (and how poor most people's passwords are).  I take a line from a song.  For example:

  Money, so they say, is the root of all evil today

Of course, now that this information is public it makes it easier for you to crack my passwords!  Even so it would take an enormous amount of time to try to crack my passwords by taking different consecutive groups of words from books and songs, even if you knew what books and songs I like.

For even more security, I also try to add some changes such as replacing "to" or "too" with 2, etc.  For example:

  $, so they say, = the root of all evil 2day

An added benefit is that the password is more likely to pass the password checking rules that require digits, punctuation characters, etc.

Summary

Here are the main points:
  • don't use passwords of one (or even a few) words in any language
  • don't use passwords related to public knowledge about yourself
  • use long passwords, if possible
  • don't use passwords that are hard to remember
  • don't force passwords to be changed too often (eg more than every 3 months)
  • don't write passwords down where they can be seen by unauthorized persons
  • don't type your password where visible to a person, camera or phone
  • change your password immediately if you think it may have been compromised

Monday, 13 August 2012

Adding Windows 7 TaskBar Support


The new MFC (10) has built-in support for the Windows 7 TaskBar.  A lot of people are using Windows 7 now so I thought I better update HexEdit to take advantage of the new features.  In particular I wanted to add support for:
  1. thumbnails that appear when you hold the mouse over a TaskBar button
  2. for progress display when HexEdit is performing a long task
  3. for the task list that is shown when you right-click the taskbar button

I describe my success with these below but first look at some general issues I first had to solve.

MFC 10 and Windows 7

The first thing you need to use the Windows 7 TaskBar is an App Id.  I just added a resource string with ID of AFX_IDS_APP_ID and MFC found it automatically.  You can actually let MFC create the App ID from other resource strings but this way I could set it to exactly what I wanted (ECSoftware.HexEdit.Free.40).

The other things you need to do to get access to all the new features of MFC is to set WINVER to 0x0601.  (It was previously 0x0501.)  By itself this will allow MFC to take advantage of Windows 7.  (I have no idea why internally Windows 7 is actually 6.1)

However, setting WINVER to a value greater than 0x501 (version number for XP) may cause some things to not work if the software is to run under Windows XP.  And I still want it to run on XP!  For example, the Windows headers make the structure NONCLIENTMETRICS 4 bytes larger for Vista and later (WINVER >= 0x0600) so I check at run-time what version of Windows is running and adjust the size of the structure before passing it to SystemParametersInfo().

Furthermore, I still want the code to build under Visual Studio 2008 (MFC 9) since not everyone has VS2010.  So some of the code needs to be conditionally-compiled using the _MFC_VER macro.  (Of course if you build HexEdit with VS2008 then you will not get the new MFC 10 features.)

Thumbnails

Some sample MFC 10 applications support thumbnails so I thought it would be easy to add.  I eventually got it working (see screen shot) but it took me a while to work out how to add this to an existing MFC application.

Eventually, I worked out that I only had to call EnableMDITabbedGroups() at startup, magically even showing multiple thumbnails if there were multiple file open in HexEdit.  Moreover, there were extra features such as options for a close button on tabs, coloured tabs, etc.  Even better the new tab system added the ability to create vertical and horizontal tab groups and easily move windows between them (like MS first added to Visual Studio 2005).

There were some problems that I needed to work out though such as getting my tab context menus to work again.  This was actually quite easy and even continued to allow the context menu to be customizable (which is actually not a feature of any of the sample applications).  Also the tip window (which shows the associated file name when you hold the mouse over a tab) worked simply by setting an option.

It was not till much later that I found a big problem.  The Next Window command no longer worked.  This command has been in HexEdit from the start and allows you to cycle through all the open windows.  It is particularly useful in keystroke macros.  Internally there was now a tab "window" at the same level as the MDI windows meaning that my Windows API call to "get next window" code failed.  This was not too hard to fix once I understood the problem.

I also noticed that the Cascade/Tile etc options on the Window menu now do not do anything at all, so I disable them if the new MDI tabs are on.  (There is an option to use the old style tabs.)

Progress

Windows 7 has a cool feature where if a program is performing a long task it can show a green progress band as a background to the taskbar button.  This allows you to go and do something else when your program is busy but keep an eye on the progress by simply checking the taskbar button.

When working with large files HexEdit can take a long time to perform some operations, so this is the perfect application for the new Windows 7 TaskBar progress feature.  To add this change was trivial since HexEdit already supports a progress bar in the status bar.  It was a simple matter to add 4 lines to the function that already handles the progress display to get the Windows 7 progress to work.

Jump List

Encouraged by how easy the previous addition was I jumped straight into adding a custom jump list.  However, I soon found that this was no easy task at all.

First I should explain that the jump list is a list of options you see when you right-click on a TaskBar button - usually this is just a list of recently opened files.  The list is retained and available even when the software is not running.  By default MFC will give you a list of recently opened files, but I wanted three sections:

  • Recently Open Files
  • Commonly Opened Files
  • Favorites (indicated in HexEdit by adding to the Favorites category)


After a lot of reading and experimenting I could not work out how to get it to work.  There are some articles on using the new CJumpList class on MS various MS websites but they all contain mistakes or are incomplete.

None of the included MFC samples demonstrated the the CJumpList class and a search of the Internet reveal no complete samples except for the one at http://neuron.tuke.sk/~maria.vircik/apw/VS2010TrainingKit-pre%20tych%20co%20chcu%20vediet%20viac/Labs/Taskbar%20-%20MFC/Source/.  But there was no way to download the complete project for this apart from tediously extracting one file at a time.  I tried various FireFox and IE add-ons such as PimpFish but all were useless for just downloading the project files.

Eventually I managed to download all the files for the sample using the Expression Web "Import Site Wizard".

With the help of this project I believe I now know more about how jump lists work and the problems I previously encountered.  One of the problems is that for a file to be added to the jump list the software must be registered to be able to open that type of file.

Conclusion

As usual MFC makes some things really easy and some things really hard.  (Most of the time things are easy once you work out how to do them which itself can be hard.)

I have added the code for the thumbnails and progress and committed the changes to the SourceForge repository.  Unfortunately these changes were added after the release of HexEdit 4.0 but you can get them try them if you want to build HexEdit yourself.  Otherwise you will have to wait till I release HexEdit 4.1.


Sunday, 27 November 2011

Clipboard

After a few people have asked for it, you can now set the default clipboard format to optionally be "hex text".  Actually there are 4 options now as explained below.  There are some advantages to the way HexEdit has always worked with the clipboard but the new options allow users to configure the program to work the way they expect or want it to work.

Moreover, copying a selection of any size to the clipboard is now supported (previously there was a limit of 16 MBytes).

History

When I first started work on HexEdit in 1997 I tried another Windows hex editor (I think it was called Hex Workshop).  When I used it to copy and paste data it had the very poor behaviour that all bytes with the value zero (ie nul bytes) were lost.  On investigation I discovered that it was copying data to the clipboard simply as text, but the Windows clipboard text format is not designed to handle binary data, which meant that some byte values were lost.

Then I discovered that you could copy and paste binary data using Visual Studio (actually it was called Developer Studio in those days).  The Visual Studio hex editor uses a special binary format that allows all byte values to be copied and pasted.  This was the obvious clipboard format to use with a binary file editor.  However, the disadvantage with the way Visual Studio worked is that you could not copy text data onto the clipboard to paste into another program such as a text editor.

HexEdit 1.0 Clipboard Formats

If you didn't know, the Windows clipboard allows you to copy the same "thing" onto the clipboard in several different formats at once.  For example, you could copy a picture as separate bitmap and vector graphics formats and even perhaps as some sort of text that describes the picture.  This allows great flexibility for programs to interact.

The obvious solution for HexEdit was that when copying data to the clipboard to copy it in two separate formats:
1. as binary data (in the format used by Visual Studio)
2. in standard text format to allow pasting into other programs

When pasting, HexEdit would use the binary data format if present, so there was no chance of any loss of data when copying and pasting within HexEdit itself.  This also meant that sharing data with the Visual Studio hex editor was simple.

If the binary data format was not present, and the text format was present, it would just paste the text data.

Text Formats

However, the above system only provided simple text support (but still much better than Visual Studio's support, which was none).

The HexEdit character area has always supported display in different character sets, so when copying to the clipboard as text it was obvious that the text should be converted from the current character set into the Windows clipboard text format (which was ASCII for Windows 95 and descendants, or Unicode for Windows NT/2K/XP/7).  For example, this allowed you to easily extract data from an EBCDIC text file and paste it into a Windows text editor or word processor which worked with ASCII or Unicode.

So HexEdit allowed you to convert when pasting into a different program but this ability was lost when copying and pasting within HexEdit itself.  That is, you could not simply copy and paste within HexEdit to convert from one character set to another since HexEdit first looks for a binary format and pastes it verbatim (ie, with no conversion).

So I added the Paste As ASCII/EBCDIC/Unicode commands to allow pasting and conversion of text.  Many users, especially those working with EBCDIC text found this very useful.

Hex Text

Another deficiency was that sometimes you want to copy the binary data as "hex text".  For example, say you have data of containing an ASCII 'A' character and an ASCII 'B' character and a nul byte.  When copied to the clipboard in Hex Edit this would have been copied as:

binary data: 0x41, 0x42, 0x00  (3 bytes)
text data:   "AB"              (2 bytes)

But sometimes you want the equivalent "hex text":

text data:  "41 42 00"         (8 bytes including spaces)

One use of this is to copy the hex values into a text editor - which some people like to do for some reason.  Another use is to copy an address from the binary data in a file as "hex text" and paste it into the Hex Jump Tool to jump to the address.

This was why I added the "Copy as Hex Text" command to HexEdit.

Paste From Hex Text

Now you would think that the above facilities would cover any possible contingency.  However, some users wanted more.  Some were used to other hex editors which took the simplistic approach of simply copying and pasting all data as "hex text".  Someone complained that since there was a "Copy as Hex Text" command there should be a corresponding "Paste From Hex Text" command.  (There is an "Import from Hex Text" command, which reads hex text from a file but no equivalent to read hex text from the clipboard.)

Apparently several other hex editors do all their clipboard interaction using "hex text".  This allows binary data to be handled without loss and also allowed pasting "hex text" into a text editor.  However, it does have a few problems:

a. It is much slower for large amounts of data due to the conversion time for both copying and pasting.
b. Uses more memory as there are at least 3 bytes stored on the clipboard for every byte of binary data.
c. You can't copy actual text data from the hex editor into a text editor, since it is always converted to "hex text".

However HexEdit Pro 4.0 provides this facility as an option.  (See Workspace/Editing page of the Options dialog.)  Actually, there are now four options:


  1. As HexEdit has always done it (binary data and text data)
  2. Always use "hex text"
  3. If the data appears to be text then use option 1 above, else use option 2.  This allows copying of actual text data so it can be pasted into something else.
  4. When the selection was made in the "character" area (to right of hex area) use option 1 above, else (if selection is in the hex area) use option 2.

I think the above options should cover just about anything, but if you have any feedback on improvements they are most welcome.

Large Selections

However, this is not the only improvement to clipboard use in HexEdit Pro 4.0.  One problem that a small number of users have encountered is when working with very large amounts of data.  The problem is that Windows limits the amount of data that can be placed on the clipboard.  (Under Windows 95 this was 16 MBytes, but under Windows 2000/XP/7 the maximum size varies depending on the amount of RAM in the system but is usually much less than 100 MBytes.)

When placing large amounts of data on the clipboard HexEdit Pro 4.0 uses a custom clipboard format, which uses a temporary file on disk to store the actual data.  All that is placed in memory on the clipboard is the name of the data file.

This facility allows copying and pasting blocks of unlimited size, limited only by available disk space in the user's temporary data area.  This has been tested with a selection of 100 GigaBytes, which is more than 1,000 times bigger than could previously be handled.  Of course, it takes a bit of time to copy this much data around on disk.

Monday, 17 October 2011

Bookmarks

If you have ever had to maintain a large, poorly designed piece of software then you know that even a trivial change can take a long time to make.  The problem is not actually making the changes to the code but finding what to do and often most of the time might be actually finding exactly where to make the change.  Personally, I have often spent hours, even days, trying to find something but once found the actual code change might only take a few minutes.  (Note that I am not actually talking about HexEdit here though I admit that the source for it can suffer from this problem as it is poorly designed in places.)

VC++ Named Bookmarks

Back when I first started using the Microsoft C++ compiler (VC++ 4.1, I think) I found the named bookmark system absolutely indispensable for keeping track of important places in the code.  I also used it to keep track of all bugs that I fixed by prefixing the bookmark name with a bug number - that way I could easily find the relevant code if I had a similar or related bug.  I liked it so much that I modelled HexEdit's bookmark system on the Visual Studio system.  (Actually I think the VC++ IDE was called Developer Studio, not Visual Studio, in those days.)

The things that made the bookmark system particularly useful were that bookmarks were persistent, project-wide and you could assign each bookmark a meaningful name.  You needed to know nothing but the name of the bookmark to be able to jump straight to the source code.  You could even jump to a bookmark in a file that was not even open and the IDE would kindly open it for you.

[The worst thing was that the bookmarks were stored in a binary format and you could easily lose all your bookmarks if the file became corrupted, which often happened.  I quickly learnt to backup the bookmarks file regularly but not being text it was not amenable to being placed under version control.]

As I said, I modelled the HexEdit bookmarks on the VC++ 6 (DevStudio) bookmarks system but there were also some improvements...

Improved Dialog

First I made the bookmarks dialog modeless so you could leave it open to always be able to see the list.  This also makes it simple to jump to a bookmark (just by double-clicking it) and it's also much easier to add a new one.

It is just stupid for a dialog that contains a list to not be resizeable, especialy as monitors are always becoming of higher resolution.  I always found the pokey little Dev Studio dialog to be annoying.

Finding Bookmarks

The HexEdit bookmarks dialog supports several columns of information which can be resized and even hidden.  Sorting on a particular column can make it easy find the bookmark you are looking for.

One thing I find annoying is when you can't remember the name of a bookmark.  HexEdit also keeps track of when each bookmark was last modified (created or moved) and when it was last accessed, or jumped to.  This information can be useful to find a bookmark you can't remember the name of.

Fixing Annoyances

It's annoying when you go to jump to a bookmark but the file it points to no longer exists on disk.  HexEdit provides a Validate button so you can check that all the bookmarks are valid.

I also got rid of the silly idea of having completely separate lists of named and unnamed bookmarks.  In HexEdit if you create an "unnamed" bookmark (eg by pressing Ctrl+F2) it adds a bookmark prefixed with "Unnamed" such as "Unnamed001".


One annoyance I encounter when using HexEdit is that bookmarks mysteriously seem to move.  (And no, it is not a bug in HexEdit).  I suspect the most common reason is that the file has been modified outside of HexEdit rendering the bookmark slightly out of place.  As a way around this I recently added a very useful feature -- if you see a bookmark is in the wrong place you can just click and drag it to the right place.

Bookmarks Tool

Before I forget I should mention the bookmarks tool.  This is a drop list on the standard toolbar which shows all the bookmarks in the active file.  As you move around in a file the tool updates to shows the closest bookmark above the cursor.  What I find really convenient is the ability to select a bookmark from the list to quickly jump to it.

Visual Studio Bookmarks

When VS.Net (aka Visual Studio 2002) was released Microsoft rewrote the IDE and accidentally or intentionally removed almost all support for bookmarks.  This was a major step backward from VC++6 so obviously I wrote an email of complaint.  I also made a list of suggestions in the same or another email including most of the facilities in HexEdit:
  • modeless, resizeable, dockable bookmarks window
  • sortable list with multiple columns
  • Ctrl+F2 (unnamed bookmark) should just create a special name and add it to the list of named bookmarks
  • bookmark categories
I don't know if anyone even looked at my emails but since then Visual Studio has added all these features and a few more.  Though the features are good the implementation sucks but discussion of that may have to wait for another post.

Wednesday, 28 September 2011

Recent File Lists

Have you ever gone to open a file from your recent file list and find that it has gone?

This has happened to me often (in Word, NotePad++, and other programs) and it is extremely annoying especially if you can't remember where the file is stored.  The trouble is you come to rely on being able to open a special file (or several files) from the list and consequently forget where it is on the disk.  Then it takes a lot of time to go and find it.

For example, just now I went to open an Excel spreadsheet with a table of credit card numbers (test numbers not real ones for those of you who were wondering).  Because I had also been recently working with some test scripts in Excel spreadsheets the list of about 20 files in the recent file list did not include the file I needed anymore.  I searched my local disk and could not find it, and I really didn't have the time to search our large network drive.  (Luckily, I found a copy attached to an old email.)

This is why I find it surprising that I have not seen any other software copy my idea of the "Recent File Dialog" which has been in HexEdit for almost 10 years.  It basically keeps track of all the files you have ever opened (as well as the time you last opened it in HexEdit and other things), until you delete them from the list or clear the list completely.

If anybody knows of other software that has copied HexEdit's approach please let me know!

Windows 7

I guess the above problem is the reason that Windows 7 has added the facility for frequently used file lists.  For software that supports it you can right-click the program icon on the Taskbar and see a list of frequently used files (as well as the familiar recently used file list).  I applaud this improvement but there are additional advantages to the system provided in HexEdit ...

What if you have forgotten the name of the file but remember (or can work out) when you created it, modified it or last opened it.  You can sort the list of files in HexEdit on any column such as time created or modified, size, or the time you last opened it.

You can also use the category, keywords and comments fields (since HexEdit 3.2) to make it easy to find the files you are looking for.  See this forum post for more: http://hexedit.com/cgi-bin/ikonboard.cgi?act=ST;f=6;t=2.

Deleting Entries in the Recent File List

Another annoyance I just noticed with Excel (2007 version) is that you can't delete from the recent file list.  I right-clicked on a file in the list and, though a popup menu appeared, there was no option to remove the file from the list.

You may ask why would anyone want to do this?  There are many reasons:
  • to remove a file from the list that you know you will never open again
  • to remove a file that you know has been moved or deleted
  • to remove a file that you don't want anyone to know you opened
  • to remove unimportant files so that the list is not cluttered
This is easy in the HexEdit Recent File Dialog.  Just select the file or files and click the "Remove" button.

You can also have HexEdit automatically remove files from the list is they no longer exists on disk - for example, if they have been deleted.

Monday, 19 September 2011

When is a Leak not a Leak?


I received an email recently pointing out that HexEdit has memory leaks.  If you build and run the debug version of the new HexEdit 4.0 beta then when the program exits the debug heap spits out many messages about memory "leaks".  I have tracked down several of these "leaks" and it turns out they are not really leaks but one-off allocations for singleton objects.  This is not really a leak but better described as a "splash".

I think it is pretty poor of the MS CRT (Microsoft C run-time) to pronounce all non-deallocated memory as "leaks", especially as the CRT itself does one-off allocations (eg for the FILE table) but uses a special flag so that it's own "leaks" are hidden.  Often it is harmless or even desireable to not deallocate one-off objects.  But when the CRT debug heap uses the word "leak" it seems to create panic in some people who should know better.

I am pretty diligent about avoiding memory leaks.  Here I define a leak as a continual loss of memory during the running of the software (or some part or parts of the software) that eventually will result in memory exhaustion.  Typically this occurs when a function allocates an object on the heap but forgets to free it when it returns to the caller, leaving an unused but allocated block of memory.  Every time this function is called more memory is lost.

In modern C++, using auto_ptr and the like, it is fairly easy to avoid accidentally doing this.  Some of HexEdit was written before the advent of auto_ptr though; nevertheless I believe there are no such leaks in HexEdit.

My point is that I am not too bothered about all the messages about leaks from the debug CRT.  At various times I have tracked down many of these and they have always turned out to be harmless splashes.

However, I have worked in teams where the managers insists that all such "leaks" are fixed.   And if you have developers who are prone to introducing leaks then that is probably a good idea.  That way, when you do get a real leak it is not hidden amongst all the other bogus "leaks" in the debug CRT leak report.

On the other hand I have seen software which allocates a lot of global data (caching large amounts of data for performance reasons).  Insisting that all the objects on the heap are freed at program exit can mean that the program takes several minutes to exit.  In this sort of situation it is simpler and faster to simply exit and let the CRT/OS tidy up.

VLD

Anyway, I will endeavour to tidy up all the "leaks" in HexEdit.  Luckilly, there is a library that makes this easy, often trivial, to do.  See http://vld.codeplex.com for more information.

If you have ever used the MS CRT debug heap to try to track down memory leaks you know that it can be extremely tedious, sometimes nigh impossible.  The problem is that, although it tells you where the allocation was made, it tells you nothing of how the flow of control got to that specific allocation.  For example, there could be thousands of CStrings used in a program and there may be just one that was not deallocated.  Sometimes the contents of the memory (eg the text in the string) may give a clue but not always.

The beauty of VLD is it provides a "call stack"  that allows you to very quickly track down the problem piece of code.