Todd Schiller - web browsers

A brief history of browser extensibility

2021-03-04T00:00:00-05:00

Web browsers are the modern operating system. Google's Chrome OS, an operating system that runs the Chrome Browser exclusively, is now the second most popular desktop operating system. Additionally, the rise of Electron and similar cross-platform frameworks means that many of the applications we use every day (e.g., Slack) are actually browser applications in disguise.

The ubiquity of browsers has cemented their critical role in consumer productivity and self-efficacy. Given this importance, I wanted to learn how browser extensibility has evolved over the years. Since I couldn't find a complete history, I set out to write my own.

Disclaimer #1: I'm going to gloss over many details to keep this post as non-technical as possible. I won't, for example, dig into the distinction between a browser and a rendering engine.

Disclaimer #2: If I make any mistakes or omit anything you feel is important, please post in the comments. As I have not personally had much exposure to browsers in non-English-speaking countries, there will undoubtedly be gaps in this post.

Consumer Web Browsers (1993-)

In the beginning, there was Tim Berners-Lee's WorldWideWeb browser (1990). However, Marc Andreessen's Mosaic (1993) was the first browser to gain mainstream popularity, as it was easy to install, displayed graphics, and provided customer support. Andreessen then went on to launch Netscape, which would battle Internet Explorer (first released in 1995) for dominance in the browser wars of the late '90s. The browser wars culminated with Microsoft facing antitrust action over its integration of Internet Explorer and Windows.

Plug-ins (1995-2015ish)

The early web only supported static content. Plug-ins -- native executables that render dynamic content in the browser -- opened the door for multimedia, enterprise applications, and gaming. Internet Explorer supported plug-ins via its ActiveX technology, while other browsers supported NPAPI.

On the consumer side, you basically couldn't browse the web without installing the Flash (1996) and Java (1998) plug-ins.

As native executables, plug-ins were a chronic source of security vulnerabilities and instability. Despite these problems, it wasn't until the adoption of the HTML5 standard and modern Web APIs in 2014 that browsers could rid themselves of plug-ins without sacrificing functionality.

Desktop browsers began dropping support for plug-ins in 2015, with Chrome leading the charge. On mobile, iOS (2003) famously did not offer Flash from the outset.

The Flash plug-in has been the longest holdout due to the ubiquity of Flash content. Firefox, for instance, refused to drop support for Flash until the beginning of this year.

User Style Sheets (1998-2019ish)

With multiple browsers on the market and growing incompatibilities in site rendering, Tim Berners-Lee founded the World Wide Web Consortium (W3C) in 1994 to develop cross-browser standards.

One of these standards was 1996's Cascading Style Sheets (CSS) specification. CSS defines support for "stylesheets" that tell a browser how to style the content on a page. Stylesheets can come from three different sources:

Document author: The author of the website
User: The user of the browser
User agent: The default style for the user agent (i.e., the browser)

In 1998, Opera 3.5 was the first browser to support CSS and included support for user stylesheets. Internet Explorer followed suit with the release of Internet Explorer 4 in 1999.

In 2005, with the introduction of Jason Barnabe's Stylish extension, consumer styling began shifting to style manager browser extensions. Compared to user stylesheets, style managers offer more flexible controls, sharing features, and ease of use.

Chrome dropped support for user stylesheets in 2014. Firefox still has support via the userContent.css stylesheet, but the user stylesheet hasn't been accessible from the main interface since 2019.

**Correction 6/5/2021: Safari still supports a user stylesheet from the Advanced tab in preferences. Here's a history of custom stylesheets in Safari. **

Bookmarklets (1998-)

Brendan Eich and Netscape introduced JavaScript to the world in 1995. JavaScript is a programming language that website authors can include to create dynamic content and interactive user interfaces.

JavaScript also opened up a new avenue for browser customization: bookmarklets. Bookmarklets are small snippets of JavaScript contained in a bookmark that starts with javascript:. When you click the bookmark, it runs the JavaScript in the context of the page, possibly loading more JavaScript from another source.

For example, here's a bookmarklet that opens the historical versions of a page in the Internet Archive's Wayback Machine:

javascript:void (window.open('https://web.archive.org/web/*/' + location.href.replace(/\/$/, '')));

Bookmarklets hit a road bump in 2004 with the introduction of W3C's Content Security Policy (CSP) specification. Designed to protect sites against malicious user content and third party code, CSP enabled site owners to lock-down code origins and network requests.

However, wanting to maintain extensibility, the CSP Specification specifies that bookmarklets and other extensibility features should be exempt from CSP restrictions:

Policy enforced on a resource SHOULD NOT interfere with the operation of user-agent features like addons, extensions, or bookmarklets. These kinds of features generally advance the user's priority over page authors, as espoused in [HTML-DESIGN].

In practice, things are different. In Firefox, bookmarklets are subject to the site's CSP, preventing bookmarklets from being run on many pages. In Chrome, bookmarklets aren't subject to the CSP. But, by default, Chrome only shows bookmarks on the New Tabs page; users must configure the bookmarks bar to be shown on web pages.

Browser Extensions (1999-)

The modern approach to browser extensibility, browser extensions, kicked off in 1999 when Internet Explorer added support in Internet Explorer 4. The new extensions supported "Explorer Bars" (i.e., the toolbar) and added entries to context menus.

Browser extensions, also sometimes called add-ons, support modification of the browser interface, modification of a webpage's user interface, and running JavaScript in the context of a webpage. These days, browser extensions support a wide variety of use cases ranging from writing assistance and web coupons to adblocking.

Firefox launched with extension support in 2004, with the other mainstream browsers following suit in 2009-2010. As early as 2005, the "Mozilla Update" page was bustling with activity:

The major browser vendors each maintain their own browser extension marketplace where extensions undergo a security review and updates can be automatically distributed to users.

Unfortunately, because browser extensions have access to a user's browsing activity, they're an attractive attack vector for malicious actors.

To curb abuse, Chrome, in particular, has taken steps to restrict how browser extensions are distributed. In 2014, Chrome introduced restrictions so that Windows users could only install extensions from the Chrome Web Store.

Mozilla XUL (1997-2017)

Mozilla and Firefox's user interface was implemented using a language called XUL, the XML User Interface Language. The XUL defines a user interface that is rendered on the fly.

In Firefox, browser extensions could modify the XUL, thus allowing them to change core parts of the user interface (e.g., adding toolbars).

Firefox discontinued XUL for extensions in 2017 because it led to vulnerabilities and instability when multiple extensions modified the UX. Technical users can still make limited modifications to the Firefox interface via its userChrome.css settings file.

Alternative Browser Distributions (2004-)

The Netscape browser was open-sourced in 1998 and used to build the Mozilla browser suite. Mozilla Firefox, the first mainstream open-source browser, was subsequently released in 2004.

In 2008, Google released the Chromium open-source browser project. Google uses Chromium as the basis for its Chrome web browser. By being open-source, developers can "fork" the browser, modify it, and distribute their own custom versions.

There have been a lot of alternative browsers, many of which have not stood the test of time. Some notable examples include:

GNU IceWeasel/IceCat (2005-): Firefox distribution without proprietary plug-ins.
Flock (2005-2011): Browser with integrated social media features. Originally based on Firefox and later based on Chromium. Was purchased by Zynga, a social games company.
Tor Browser Bundle (2008-): Firefox distribution that makes use of the Tor network for private browsing.
Brave (2016-): Brendan Eich's (of JavaScript fame) Chromium browser with content-creator revenue sharing via the Basic Attention Token cryptocurrency.
Microsoft Edge (2019-): Microsoft's Chromium-based browser after their in-house browser (also called Edge) failed.

Userscripts (2005-)

Userscripts are small scripts that modify a webpage. Like browser extensions, they're typically written in JavaScript. However, userscripts are lighter weight in that you don't have to package them and distribute them through the browser's web store. Instead, to run and manage userscripts, you use a browser extension aptly called a userscript manager.

Userscripts provide many benefits over bookmarklets: they can run automatically on a page, aren't subject to length restrictions, and can run outside of a site's CSP.

The first userscript manager, Greasemonkey, was released for Firefox in 2005. There are now several variations to choose from, including Tampermonkey (2010) and Violentmonkey (2013). These variations all use the same Greasemonkey-style scripts.

In 2019, Firefox added formal support for userscripts in extensions, providing additional security guarantees. Chromium has a separate sandbox feature to run code in isolation, but has no formal userscript support. However, as I will touch on a bit later, the end is nigh for userscripts in Chromium browsers.

Converging on the WebExtensions API (2017-)

Abandoning XUL-based extensions in 2017, Firefox switched to the defacto standard WebExtensions API by and large mimicking Chromium's API.

Firefox's shift means that the vast majority of browser extensions are now using the APIs defined by Chrome's Manifest Version 2, first released way back in 2012 (although additional incremental API functionality has been introduced since then).

Due to its waning market share, Firefox chose to support the chrome API namespace in addition to the vendor-agnostic browser API namespace. In many cases, Chrome extensions can be ported over to Firefox with no code modifications. Firefox also provides a polyfill for developers to make their vendor-agnostic code compatible with Chrome.

Writing extensions for Safari traditionally involved writing them using Apple's XCode development tools. Starting with Safari 14 in 2020, Safari also adopted the WebExtensions API.

Manifest V3 (2021-)

In January 2021, Chrome released its controversial Manifest Version 3 (V3) of the WebExtensions APIs. "Manifest" refers to both the technical integration points and the policies Chrome enforces when reviewing extensions for its Chrome Web Store.

In line with previous browser API changes, the stated goals of Manifest V3 are security, speed, and reliability. The major changes introduced in Manifest V3 are:

Making network request modification declarative (speed/security)
Using service workers for background operations, vs. a persistent background page (reliability)
Banning remotely hosted JavaScript code, a common vector of security vulnerabilities (security)

Manifest V3 is controversial in the browser extension community because it breaks the functionality of many popular extensions.

For the general public, the most worrisome is that popular content and adblockers (e.g., uBlock Origin and Privacy Badger) will no longer be supported. These blockers work by maintaining a large ruleset to block network requests. Such large and dynamic rulesets are not supported in Manifest V3. (To be fair, in response to the outrage, the Chrome team has increased the number of rules per extension.)

In addition, userscript managers, by their very definition, work by allowing users to install and run remote JavaScript code. As of now, even user-pasted code will not be supported by Chrome once support for Manifest V2 is removed (likely in 2022) .

The Future: No/Low-Code Browser Extension Builders (2021-)

Ever since mainstream browsers added support for extensions, the developer community has built an incredible ecosystem of extensions that gives consumers control over their browsing experience:

Style managers (2005-): Customize style of a page.
Userscript managers (2005-2022?): Run custom code automatically.
Content/adblockers (2002-): Block network requests and elements.
Context menu managers (?-): Add context menu items (e.g., opening a new URL dynamically based on the selected text).
Web clippers (?-): Clip information from a page; customize what's clipped and where it's sent.

Despite the number of extensions published (the Chrome Web Store has 189K extensions), there is still a lot of room to innovate. In particular, the pairing of advances in no-code + API-fication with browser extensions has the potential to increase consumer productivity and self-efficacy dramatically.

To realize this vision, I founded PixieBrix. At PixieBrix, we are working to create an open-source browser extension for consumers and companies to customize their browsing experience to fit the way they work. PixieBrix is no/low-code -- you configure and combine "bricks" to add missing functionality, integrate sites without APIs, and automate repetitive tasks.

Personally, I'm very excited by the surge of interest in this space. One notable startup I've come across is Insight Browser, an iOS browser that allows users to build and share simple extensions. In the enterprise world, Extension.dev is another low-code builder for internal enterprise extensions (check out their comprehensive "What internal extensions are people building?" page).

If you're interested in working or investing in this space, I'd love to hear from you. Want to follow us on our journey? Follow us on Twitter, star our GitHub repository, or subscribe to our newsletter.

7 bite-sized tips for reliable web automation and scraping selectors

2020-12-03T00:00:00-05:00

If you're like most developers, you've probably encountered Cascading Style Sheet (CSS) selectors for styling webpages.

For example, the following CSS rule combines a paragraph element selector p with a class name selector .lead to set the font size:

p.lead {
    font-size: 32px;
}

We can mix-and-match CSS selectors to describe any subset of elements on a page. There are CSS selectors for HTML tag types, ids, classes, attributes, page structure, and even UX interactions.

Because of their expressiveness, CSS selectors are used everywhere in the web ecosystem:

Web Applications (Web API, JQuery): modifying the DOM, attaching event handlers, etc.
Automated Testing: finding elements to interact with, checking assertions, etc.
Web Scraping: selecting data to extract, finding links to traverse, etc.

Over at PixieBrix, selectors are an integral part of our web customization engine.

Unfortunately, if we don't control a web page, writing reliable selectors can be an unpleasant experience. Some old-school WYSIWYG-built sites (I'm looking at you, Microsoft FrontPage) and modern Single Page Applications (SPAs) can be downright hostile to work with.

To help folks out, I've compiled a set of tips I find useful when writing selectors for enhancing, automating, and scraping 3rd-party sites.

Tip #0: Use JQuery extensions

When working in a browser context (normal or headless), use JQuery's selector extensions. Selectors such as :eq , :header , and :contains make many selections significantly more straightforward.

A downside is that JQuery extensions are slower because they can't leverage the browser's native CSS evaluation. However, for automation and scraping, you probably won't ever notice. If it does become a problem, there are simple techniques to optimize JQuery selectors.

In addition to CSS+JQuery, it's also helpful to learn a bit of XPath, an XML/HTML traversal language that browsers support natively.

Tip #1: Avoid structure-based selectors

Browsers and some libraries are able to automatically generate unique CSS selectors for elements.

Automatic generators are great because they can take advantage of the structure of the Document Object Model (DOM). Automatic generators are also terrible because they sometimes rely too much on the structure of the DOM.

Recently, a tool gave me the following selector:

:nth-child(5) > :nth-child(2) > :nth-child(2) > :nth-child(1) > :nth-child(3)

Structural selectors like these are extremely sensitive to small changes on the page, including non-visible changes. (For example: an element with display:none could be inserted before an element.)

These selectors will silently return the wrong element, and we'll wind up debugging a failed automated test, or ingesting bad extracted data.

Therefore, whenever possible, I eschew overly-specific structural selectors in favor of ids, class names, attributes, and textual content.

Tip #2: Avoid dynamically generated attributes

In modern Single Page Applications (SPAs), element ids and class names are commonly dynamically generated.

Dynamic class names are computed when the code is compiled/bundled. That's because the class names in the HTML file must match the names in separate CSS stylesheets. Dynamic class names will change whenever the site is updated. ( Unless they are computed deterministically, e.g., using a hash. Then they'll change whenever the style is changed).

For example, take the "Google Search" button on the Google homepage. It's built with Google's Closure Framework:

<input class="gNO89b" value="Google Search" name="btnK" type="submit">

At first glance, the name="btnK" also looks random, but it's not. The "I'm Feeling Lucky" button button on the page has the name btnI, suggesting they're human-picked. As a rule of thumb, input name attributes can't be dynamic because other parts of the application (either front-end, or backend) depend on them.

A common source of dynamic class names in applications is the use of CSS Modules. CSS Modules automatically encapsulating styles to avoid accidental styling collisions. Here's an example header:

<h1 class="_styles__title_309571057">An example heading</h1>

While we can't match the exact class name, it will follow a consistent naming pattern across updates of the site. Therefore, we can match on the pattern using CSS's starts-with attribute selector:

h1[class^="_styles__title_"]

Dynamic ids, on the other hand, are managed by the SPA framework at runtime. Therefore, they'll change for each run of the application. Take, for example, the profile header from LinkedIn, which uses the Ember.js framework:

<section id="ember1398" class="pv-top-card artdeco-card ember-view">
    <!-- more elements -->
</section>

Here, the ember1398 id isn't random — the framework sequentially generates a new id for each element it renders. In practice, the id of an element will differ between page loads because elements don't always load in the exact same order.

Tip #3: Search text with JQuery's `:contains` selector

JQuery's contains selector selects elements that contain the given text anywhere within them. The selector is not available in plain CSS, but it is available as an XPath function.

For example, consider the following HTML:

<div>Lorem ipsum dolor sit amet, consectetur ...</div>

We could select this element with:

:contains("Lorem ipsum")

If there are other tags, we need to add additional selectors to clarify which element we want. For example, the above selector would select the div, p, and b elements in this document:

<div>
    <p><b>Lorem ipsum</b> dolor sit amet, consectetur...</p>
</div>

To select just the div, we'd need to specify the tag in the selector:

div:contains("Lorem ipsum")

In some cases, the text might contain HTML entities, e.g., a non-breaking space:

<div>Lorem&nsbp;ipsum</div>

Trying to select "Lorem ipsum" won't match here. In these cases, we provide the contains selector multiple times to match only elements that contain both words:

div:contains("Lorem"):contains("ipsum")

Finally, a caveat/warning: when writing text selectors, be aware of which text is translated when the page is viewed in a different language.

Tip #4: Target selection with `:has`

Sometimes we need more precision in targeting a text search. In these cases we can combine a :contains selector within a CSS :has selector. Suppose we wanted to get the year a property was built from Apartments.com:

<div class="specList">
    <span><h3>Property Information</h3>
      <ul>
         <li><span class="bullet">•</span>Built in 2018</li>
         <li><span class="bullet">•</span>1016 Units/44 Stories</li>
      </ul>
</div>
</div>

We can find the div with the Property Information header, and then grab the list item corresponding to the year the property was built:

.specList:has(h3:contains('Property Information')) ul > li:contains('Built')

The :has selector can also be used to find the containing element. The following selector matches a list containing an item with "Built", rather than the individual list item:

ul:has(> li:contains("Built"))

Tip #5: Use ARIA attributes for more readable selectors

Accessible Rich Internet Applications (ARIA) attributes support the accessibility of a site, e.g., for providing context to screen readers. Using them for selectors also makes selectors more readable.

For example, the aria-label attribute labels elements that otherwise would depend on visual cues:

<button aria-label="Close" onclick="myDialog.close()">X</button>

These can be selected against using standard CSS attribute selectors, e.g.:

button[aria-label="Close"]

One thing to watch out for with the aria-label attribute is that it's always internationalized/translated alongside other UX text.

ARIA attributes also provide information about the role of elements on the page (e.g., the role attribute) and even the relationship between elements on the page ( e.g., aria-labelledby).

Tip #6: Be on the lookout for automated testing attributes

Developers using automated testing frameworks, e.g. Cypress, add data attributes to make their life easier when writing automated tests. Sometimes these attributes will find their way to the production site:

<div data-test-id="content"><!-- some content --></div>
<div data-cy="content"><!-- some content --></div>

Testing attributes can be selected just like any other attribute, and are often unique (just like an id):

[data-cy="content"]

Bonus Tip: handle flat key/value pairs with the adjacent sibling combinator

The adjacent sibling combinator (+) matches the element immediately following another element.

Supposed we have the following HTML, structured as multiple key-value pairs per row/list item:

<li class="row">
    <span class="k">Name:</span>
    <span class="v">Laura Smith</span>
    <span class="k">Year:</span>
    <span class="v">2013</span>
</li>

We can look up the value for the year by finding the "Year:" label and then selecting the next element, which will be the value:

span.k:contains('Year:') + span

Also see the Hacker News discussion.

The PixieBrix manifesto: web customization for the masses

2020-09-02T00:00:00-04:00

My freshman year of college, I interned as an "equities analyst". What this meant in practice was that I was supposed to look up numbers in a terminal, and then type them into an Excel spreadsheet. The software provider hadn't gotten around to implementing export for the data yet.

Not wanting to spend a whole summer entering data, I set out to put my computer science classes to use. I coded up a flow where I took screenshots, passed them through off-the-shelf OCR software, and then had a script add them to the spreadsheet. The two-month project took one week, and I spent the rest of the summer enjoying World Cup Soccer matches on the office TV.

That's when I realized, given the right tools, even basic coding skills can translate into massive productivity and quality of life gains.

Now almost 15 years later, I see a mixed landscape. Improvements in development tools and AI are creating new possibilities. However, non-devs are often worse off. Perpetual licenses and open-source software have given way to subscriptions and walled gardens. Software providers artificially limit the productivity of their software by designing for the average user. Or worse, intentionally restrict functionality.

We started PixieBrix because we believe three things:

There is no such thing as an "average user"
Everyone should be empowered to customize their software to give them superpowers
Computing should be delightful

To this end, we've set out to create low/no-code tools for customizing the behavior of software, web pages, and SaaS. But we won't be able to do it alone. That's why we're also working hard to build a diverse and vibrant community of makers.

Our first PixieBrix product is a low/no-code platform for extending web browsers. Web browsers are the new OS, and everyday people are constantly switching between countless web pages and SaaS apps. Now, with a bit of configuration, anyone using our platform can add functionality that previously required custom browser extension development. This ranges from adding elements to a page, to automating workflows à la robotic process automation (RPA).

Currently, building new features in our system requires common coding skills such as YAML, CSS selectors, JSON Schema, and regular expressions. Over time, we'll be working hard to reduce the barriers to use, mix, match, and create.

Fitting with our beliefs, I'm proud to say our browser extension is open source (GPL v3). That means everyone can inspect what code they're running and modify it, as long as they're willing to extend that benefit to others.

I'd like to personally invite you to join us on our journey. To keep up to date, subscribe to our newsletter. And, if you have an idea you'd like to build, send me an email at todd@pixiebrix.com. I'd love to hear about it and help turn it into a reality.