BFiles Extension for Mercurial

The BFiles extension is a plugin for Mercurial that allows you to more efficiently manage Big files and Binary files.
For more on this problem see this link Binary Files and Mercurial

Documentation:

Installation and Configuration

1. Download and unzip the BFiles Extension:

  • http://mercurial.selenic.com/wiki/BfilesExtension

2. Edit your .hg/hgrc file or %HOME_DIR%/mecurial.ini file to configure the extension:

[extensions]
bfiles = C:\bfiles_MecurialExtension\bfiles

[bfiles]
store = C:\bfileRepo
autostatus  = true
autoupdate  = true
autorefresh = true
autoput     = *

bfiles

  • Location of where you unzipped the extension (in my case C:\bfiles_MecurialExtension\bfiles)

store

  • Location of where you want the actual binary files stored (in my case C:\bfileRepo…but most times this is likely going to be some network location you can access using a url and ssh…see Usage Doc)

autostatus

  • “If true: modifies the “status“ command to show the status of big files mixed in with regular files. This makes big files look like regular files”

autoupdate

  • “If true: modifies the “update“ command to run “bfupdate“ after updating normal files.”

autorefresh

  • “If true: modifies the “commit“ command to run “bfrefresh“ before committing.”

autoput

  • “If set, modifies the “push“ command to run “bfput“ before pushing to certain repositories.”
    • In my case http://myCentralRepository:8000/* (NOTE: THE * for reg exp)

Notes:

  • Run the hg bfadd <FILENAME>command seems to add a big or binary file to be handled by bfiles instead of Mercurial directly.
    • NOTE: So far I could get this to work with a new file that had not been committed to Mercurial, but not with existing files
      • Bit of a pain but to solve this you do the following:
        1. Backup zip in question.
        2. Delete the zip -> commit -> push (now it's out of Mecurial tracking)
        3. Copy the backup to the original folder.
        4. Run hg bfadd <FILENAME> to indicate to track via bifgiles
        5. Commit -> push (NOTE: It will look like you are committing and pushing the actual zip but you aren't see next step)
        6. Check the repo you pushed to and you should only see the .hgbfile directory has changed (no sign of you actual zip)
           - The zip will have been pushed to the repo as long as you have the auto properties configured in ~/mecurial.ini or ./hg/hgrc
  • This creates a folder called .hgbfilesthat will be pushed and managed directly with Mercurial
    • This file contains a file with the “Hash” of the binary file at that moment in time.
      • For example if you compare two different changesets where the file has changed you would see something like the following in the .hgfiles directory:
        73ef2cb3d1f6e7f513342444a29e84fc2a7e55a6
        
        versus
        
        dae6a7a3ad905bf47c2cce5480f0e273d1cfc3c8
  • In addition the .hgbfiles acts like .hgignore and the management of the actual big / binary file is now ignored by Mercurial.
  • You still need to commit and push. Once you do that….
    • The above hash corresponds to the actual binary file located in your repository (in my case C:\bfileRepo)
    • When you update to a particular changeset bfiles extension will look at the hash, go get the corresponding file in the repository and relocate it locally in your workspace.

Problem with TortoiseHg

There are some Issues with TortoiseHG

  • On the command line if I update between revisions everything with the bfiles configured as above seems to work.
    • For example if I update between two revisions the binary file swaps as I would expect.
      hg -v update -r 10
      
      hg -v update -r 11
      • Remember given the above config Bfiles overrides the update command to run bfupdate after an update
  • Unfortunately the same does not hold true for TortoiseHG
    • In that case I get the following error:
      abort: uncommitted changes in .hgbfiles
      • I tracked down where this message is getting thrown from the Bfiles python code (bfcommands.bfupdate() method -> calling bfutils._check_standins_status
      • Here’s the email I sent to the bfiles extension author (greg at gerg dot ca):
        Hi Greg,
        
        Not even sure if this will get to you but assuming the email is correct on http://mercurial.selenic.com/wiki/BfilesExtension it should be ok.
        
        I'm trying to figure out why TortoiseHg Aborts with "uncommitted changes in .hgbfiles" when I try updating between two changesets with a different version of a bfile that the bfiles extension is tracking.
        
        It works on the command line but not in Tortoise, but I'm not sure why.
        
        I can get around the issue in Tortoise if I comment out this check which is called from bfupdate()
        
        def _check_standins_status(ui, repo, matcher):
            (modified, added, removed) = repo.status_nobfiles(match=matcher)[0:3]
            if modified or added or removed:
                raise util.Abort(_('uncommitted changes in .hgbfiles'))
        
        I believe it's checking to see if my file in /.hgbfiles has changed from what's in /.hg/bfiles
        - I'm not sure about this because I can't find the source for that method repo.status_nobfiles()
        - It definitely is true the hash is different in those two files, because we just updated the version in .hgbfiles and haven't run the bfupdate yet.
        
        I was thinking you might know why this is working on the command line and not in tortoiseHg? From what I see it should?
        
        Thanks,
        Chris Landry
      • Here’s what I commented out:
        C:\bfiles_MecurialExtension\bfiles\bfcommands.py method bfupdate()
        
        Line 384
        
            # Check status of standins (.hgbfiles/...) relative to main dirstate
            # (i.e. detect changes to big files that have been bfadd'ed or
            # bfrefresh'ed, but not committed).
            #bfutil._check_standins_status(ui, repo, matcher)
        C:\bfiles_MecurialExtension\bfiles\bfstatus.py method reposetup()
        
        Line 53 & 54
        
                #if not (isinstance(method, types.MethodType) and
                #        method.im_func is repo.__class__.commitctx.im_func):
      • Tests ok in Mercurial Eclipse as well.
  • Pain to take an existing binary file that is being tracked by Mecurial already and make it tracked by Bfiles
    • See NOTES section above on how to solve this.
  • Also an issue with using a network drive as your store for the binary files (what if you have multiple users (user A can’t access a folder created by user B) you have to fix this or you’ll run into issues.
  • If a client without the Bfiles extension pulls the repository in, the binary files will not be loaded and are missing. So you’ll want to have some system in place to ensure this happens.

It's only fair to share...
Share on FacebookGoogle+Tweet about this on TwitterShare on LinkedIn

Leave a Reply