How we added a favicons to our site list

Today we've added favicons to the site list you see when you log in to Oh Dear.

Using the favicon, you quickly recognize a particular site. It also just looks nice visually.

In this blog post, we'd like to share how we achieved this.

Watch it being code up

In this stream, I live coded the favicon import. You can see our actual codebase, my thinking process, and the little mistakes I made along the way.

If you prefer reading how we go about import favicons, you can continue reading underneath the video.

Using a third-party API

Oh Dear is a large Laravel application. To handle any files, we rely on Spatie's Media Library package.

Using this package, you can associate all sorts of files with Eloquent models. In this case, the file is a favicon, and the model is the Site model used in our codebase.

Grabbing the favicon

But let's first take a look at how we grab the favicon. This tweet, mentioning third party APIs to grab a favicon, inspired us to add favicons to the list.

Google has a secret API that you can use to show *any* website's icon on your site 🐳https://t.co/pcHygeE6wd pic.twitter.com/Hv9vvSVZ4V

— James Ivings (@JamesIvings) December 2, 2020

We decided to go for Duck Duck Go's favicon service because that one is the only one that gives a 404 when grabbing a favicon for a nonexisting site. That 404 is important because we don't want to import a generic favicon.

Using Duck Duck Go's favicon service is easy. All you need to do is mention the domain inside the URL of the API. Here's an example where we grab the favicon for Oh Dear.

https://icons.duckduckgo.com/ip3/ohdear.app.ico

Configuring spatie/laravel-medialibary

We don't want to link to that URL in our views directly. If we would do that, then Duck Duck Go would receive traffic each time a site list is displayed, making us a bad internet citizen. We want to import that favicon when a site is being added to the account, and maybe once a week or month from then on.

Luckily, grabbing, storing, and displaying the favicon is very easy using laravel-medialibrary. We're not going over how you should install the package in your Laravel project; you can find instructions for that in the media library docs.

With the package installed in a project, you can prepare an Eloquent model to handle media. This can be done by adding an interface a trait to the model, in our case, the Site model.

namespace App\Domain\Site\Models;

use Illuminate\Database\Eloquent\Model;
use Spatie\MediaLibrary\HasMedia;
use Spatie\MediaLibrary\InteractsWithMedia;

class Site extends Model implements HasMedia
{
   use InteractsWithMedia;
}

The media library allows a model to hold multiple collections of media. If you have a BlogPost model, you might have a collection images with all the images that need to be displayed, and a collect downloads that holds all the files that can be downloaded for a particular BlogPost.

In our case, we need a favicon collection. Though it is not strictly required, you can define a collection on the model.

// in the site model

public function registerMediaCollections(): void
{
   $this
       ->addMediaCollection('favicon')
       ->useDisk('favicons')
       ->singleFile();
	
}

Defining a collection allows you to define some behavior on it. Using useDisk we specify that any file added to the collection should be stored with the disk with a given name. In case you're not familiar with disks, it's Laravel's way of abstracting the filesytem. In our production environment, this disk points to an S3 bucket.

The singleFile collection call will guarantee that there is, at any given time, only a maximum of one file present in the collection. When a second file is added to the collection, the first one will be deleted. This will be handy for updating the favicon. We can just add the latest one to the collection, and the older one will get deleted automatically.

A nice thing to know is that when a model gets deleted, the media library will automatically delete any associated files.

Importing a favicon for a site

Fetching a favicon from an external service might take a second, so it's a perfect fit for a queued job. Here's the AddFaviconToSiteJob in our code base.

namespace App\Domain\Site\Jobs;

use App\Domain\Site\Models\Site;
use Exception;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class AddFaviconToSiteJob implements ShouldQueue
{
    use Dispatchable;
    use InteractsWithQueue;
    use Queueable;
    use SerializesModels;

    public $deleteWhenMissingModels = true;

    public Site $site;

    public function __construct(Site $site)
    {
        $this->site = $site;

        $this->queue = 'import-favicons';
    }

    public function handle()
    {
        $url = "https://icons.duckduckgo.com/ip3/{$this->site->getDomainWithoutProtocol()}.ico";

        try {
            $this
                ->site
                ->addMediaFromUrl($url)
                ->toMediaCollection('favicon');
        } catch (Exception $exception) {
            report($exception);
        }
    }
}

In the snippet above, you can see that the media library offers a convenient addMediaFromUrl method that will download a file from a URL.

We wrap the logic in try/catch block to prevent that job from failing if Duck Duck Go is down for any reason. We don't report the error to our exception tracking service Flare, so we can still get a notification that something happened.

In our codebase, we like to put isolated pieces of logic into action classes. An action class is nothing more than a regular class, where, by convention, there is an execute method to execute it. If you want to know more about action classes, check out this blog post.

We have an action called CreateSiteAction to create a new site. It gets called from the controller that handles site creations for our web app and from our API endpoint to create a site. We can dispatch the job inside of that action.

namespace App\Domain\Site\Actions;

use App\Domain\Site\Jobs\AddFaviconToSiteJob;
use App\Domain\Site\Models\Site;

class CreateSiteAction
{
    public function execute(array $attributes): Site
    {
        // ... other steps to create a site

        dispatch(new AddFaviconToSiteJob($site));

        return $site;
    }
}

Displaying the favicon

Using laravel-medialibrary, it's easy to display the favicon. The getFirstMediaUrl($collectionName) will return an URL to the first media item in the collection.

In our Blade view that renders the site list, we can just use it:

    <img class="inline mr-1 w-4 h-4" src="{{ $site->getFirstMediaUrl($collectionName) }}" alt="favicon" />
@endif

And with that, we now display favicons for any sites that are added to Oh Dear.

Importing favicons for existing sites

Of course, we want to import favicons for the thousands of sites already monitored by Oh Dear. This is done by the following Artisan command, which is scheduled to run every month.

namespace App\Domain\Site\Commands;

use App\Domain\Site\Jobs\AddFaviconToSiteJob;
use App\Domain\Site\Models\Site;
use Illuminate\Console\Command;

class AddFaviconToSitesCommand extends Command
{
    protected $signature = 'ohdear:sites:add-favicon-to-sites';

    protected $description = 'Fetch the favicon for all sites';

    public function handle()
    {
        $this->info('Fetching favicons...');

        Site::each(function (Site $site) {
            if (! $site->team->hasActiveSubscriptionOrIsOnGenericTrial()) {
                return;
            }

            $this->comment("Fetching favicon for `{$site->label}` ({$site->id})");

            dispatch(new AddFaviconToSiteJob($site));
        });

        $this->info('All done');
    }
}

In closing

Duck Duck Go and laravel-medialibrary made it painless to import and display favicons.

If you want to see this all in action, start your free ten-day trial now.