jb’s open source blog

A look into my work and play in open source

A better ‘grep’: LXR and MXR

leave a comment »

This week as a part of our OSD600 class, we had our first interactive lab. The goal was to use the Mozilla cross referencing tool, MXR (which is based on LXR, Linux cross reference) to find out where certain elements of the Firefox code are located.

I chose to do “Add Bookmark”, as I figured it seemed basic enough. Surprisingly, I found myself wading through tons of layers of code, which I didn’t expect for such a staple function. Here were my results.

First, I searched MXR within the Firefox dir for the literal “Ctrl+D”. That got me no results, so I tried “Bookmark This Page…”, and that got me started:

“Bookmark This Page…”: Found in /browser/locales/en-US/chrome/browser/browser.dtd:

61 <!ENTITY addCurPageAsCmd.label "Bookmark This Page...">

Now I did a search for addCurPageAsCmd.label:

“addCurPageAsCmd.label”: Found in /browser/resources/content/navigatorOverlay.xul:

182     <command id="Browser:AddBookmarkAs" label="&addCurPageAsCmd.label;"
183              accesskey="&addCurPageAsCmd.accesskey;"
184              oncommand="addBookmarkAs();"/>

Hmm. A decision. Do I search for Browser:AddBookmarkAs or addBookmarkAs()? I chose the latter, the former seems to be an ID whereas the latter is a function call of some sort.

addBookmarkAs(): Found in /browser/resources/content/navigator.js:

999 function addBookmarkAs()
1000 {
1001   const browsers = gBrowser.browsers;
1002   if (browsers.length > 1)
1003     BookmarksUtils.addBookmarkForTabBrowser(gBrowser);
1004   else
1005     BookmarksUtils.addBookmarkForBrowser(gBrowser.webNavigation, true);
1006 } 

Not thinking straight, I search for BookmarksUtils.addBookmarkForBrowser — but of course, this only gets me calls to this function. Of course, I should be searching for just addBookmarkForBrowser, since it belongs to something (class? package? namespace?) called BookmarksUtils.

addBookmarkForBrowser: Found in /browser/base/content/browser.js:

1574 function addBookmarkForBrowser(aDocShell, aIsWebPanel)
1575 {
1576   // Bug 52536: We obtain the URL and title from the nsIWebNavigation
1577   // associated with a  rather than from a DOMWindow.
1578   // This is because when a full page plugin is loaded, there is
1579   // no DOMWindow (?) but information about the loaded document
1580   // may still be obtained from the webNavigation.
1581   var url = aDocShell.currentURI.spec;
1582   var title, charSet = null;
1583   var description;
1584   try {
1585     title = aDocShell.document.title || url;
1586     charSet = aDocShell.document.characterSet;
1587     description = BookmarksUtils.getDescriptionFromDocument(aDocShell.document);
1588   }
1589   catch (e) {
1590     title = url;
1591   }
1592   BookmarksUtils.addBookmark(url, title, charSet, aIsWebPanel, description);
1593 } 

Now it’s addBookmark. Getting closer..

My first instinct, searching for ‘function addBookmark’ was a dead end. I found an addBookmark() function but not belonging to BookmarksUtils. So, I searched for AddBookmarks(), sans function, and found a different way of defining functions in Javascript (inherit from ‘function’, it looks like). Neat, I didn’t know Javascript could do that! (Maybe some rainy day I should dust off my “copy” of Douglas Crockford’s JavaScript.)

addBookmark: Found in /browser/components/bookmarks/content/bookmarks.js

1793   // should update the caller, aShowDialog is no more necessary
1794   addBookmark: function (aURL, aTitle, aCharset, aIsWebPanel, aDescription)
1795   {
1796     var dArgs = {
1797       name: aTitle,
1798       url: aURL,
1799       charset: aCharset,
1800       bWebPanel: aIsWebPanel,
1801       description: aDescription
1802     }
1803     openDialog("chrome://browser/content/bookmarks/addBookmark2.xul", "",
1804                ADD_BM_DIALOG_FEATURES, dArgs);
1805   },

openDialog? All that, and I still haven’t even bookmarked a page yet! This is where things started to not go my way. Chris Tyler says Mozilla devs see “everything as a webpage”, and that chrome:// url sure drives that point home. I did a search for addBookmark2.xul — but it was not found! It’s referred to often, but not actually in the source tree?

Here was a clue:

addBookmark2.xul: Found in /browser/components/bookmarks/jar.mn

  44 *  content/browser/bookmarks/addBookmark2.xul                 (content/addBookmark2.xul)       
  45 *  content/browser/bookmarks/addBookmark2.js                  (content/addBookmark2.js)

I remember vaguely from a Java class many moons ago about JAR files, and how they should ideally have “manifests”. As soon as I saw the ‘.mn’ extension, it all clicked. addBookmark2.xul, along with the rest of the chrome, I believe, is in browser.jar. And the kicker is, as far as I can tell, MXR does not archive browser.jar.

I unzipped browser.jar into a temp dir and found addBookmark2.xul. The focus of this exercise was to drudge through code via MXR, and I have no deep-seated interest in knowing how bookmarking actually works, so I didn’t bother to go any further.

Some observations, before I close:

  • I should note what I showed above was NOT me tracing through the execution of code, and it probably isn’t the exact order of things either. I still lack a real understanding of how things like .xul files contribute to the bigger scheme of things: when debugging, I can really only think in one language at a time, and I’d think of a .xul file as data to be parsed, not as code to be executed. If I ever want to understand this, I may have to change my view.
  • Given that MXR is the “Mozilla-context LXR”, I was surprised to see a lot of the crossreferencing (read: hyperlinking on function names) didn’t work in XUL. This is probably something that is non-trivial to add.
  • I was also surprised to see MXR not archiving the contents of browser.jar. Perhaps I didn’t dig deep enough, or maybe the chrome doesn’t change often enough for it to be justified, but I doubt that’s the case. It’s probably a technical limitation of MXR.
  • I’ve known for some time that these tools exist, but I never knew their names or what they could do. Granted, even now I’ve only scratched the surface of MXR, but I do recall asking Google for things like “cvs source listing”, etc on other open source projects to get an idea of what the code looked like four CVS commits ago. At least I know what that is called now.
  • And as for the title of this post: “a better grep” — although concrete examples aren’t popping up right this moment I know I’ve done something very much like this with other open source projects to try to understand the code better. I’d do..
    grep -R -e 'some string literal' *
    

    ..inside of a source tree, and that’d give me clues on where to look. Given Mozilla’s million-plus line codebase, I can see why tools like LXR and MXR were invented – to replace grep. A question now is whether this kind of source code listing + crossreferencing is worth using on smaller open source projects, with few devs (i.e. one), and whether I’ll ever use it myself in my own projects.

  • Advertisements

Written by jbopensrc

September 30, 2007 at 10:51 pm

Posted in Open Source

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: