Making a Symfony sitemap and deploying it to Elastic Beanstalk

Posted by James on August 21, 2020

Sitemaps

Much like social metadata, distributed sites don't give you sitemap out of the box. And of course, as we know, we can't make one in plain Angular because not all crawlers run JavaScript. There's a few ways I could have done this - it looks like you can configure it with Universal, or I suppose you could write a Lambda to do it. But I've been trying to think of some ways to use Symfony, so I thought I'd have a go.

I should also note that in my day job, we've got a number of sources of content - the legacy CMS, the new headless CMS, and then a series of custom apps, all of which need to be in the sitemap, so this gives you a way to make sitemaps for multi-platform environments.

Set up your Symfony application

Install Symfony, and create a new application with:

symfony new sitemap --full

Arguably it would be better to use the skeleton build, because my application doesn't really do a whole lot.

Now create a route in config/routes (or use annotations, if you prefer):

sitemaps:  path: /sitemap.xml  controller: App\Controller\SiteMapController::sitemap

Using httpClient in your controller

You'll need to use the httpClient package in your controller, so install it with Composer. You can now make a SiteMapController in your src/Controller directory, and give it a sitemap method, like this:

  public function sitemap() {
    $url = "https://cdn.contentful.com/spaces/$this->space/environments/master/entries?access_token=$this->accessToken&content_type=story";
    $client = HttpClient::create();
    $response = $client->request('GET', $url);
    $content = $response->toArray();

    $urls = [];
    foreach($content['items'] as $item) {
      $urls[] = $item['fields']['url'];
    }

    $response = $this->render('sitemap.xml.twig', [
      'items' => $urls,
    ]);
    $response->headers->add(array('Content-Type' => 'application/xml'));
    return $response;
  }

It doesn't really do a whole lot, as you can see. We hit the Contentful API, get a list of stories, loop through them and put them into a template variable. Serving it as XML is just a case of adding the content type to the response headers.

You might notice there's some variables in the API URL, though...

Handling secrets in Symfony

We've all got secrets, and if there's one important thing to remember it's that you shouldn't publish them on the internet. The Contentful API needs a space ID and an access token (in fact they aren't particularly secret - you can see them in the calls my Angular app makes, but they only give you access to published content, so it's not exactly the hack of the century. They're a good test case, though). It took me a while to figure out how to store configuration in Symfony, but here's how I did it.

Create an .env.local file in the root of your application. This is already gitignored, so you won't be able to commit it by mistake. You can add key value pairs in there, like MY_BIG_SECRET='I really love pickles'
The part that I missed is that you then need to configure access to them. There's a file called services.yaml in the config directory, where you can bind them to variables that you can then access in your code, like this:

services:
    # default configuration for services in *this* file
    _defaults:
        autowire: true      # Automatically injects dependencies in your services.
        autoconfigure: true # Automatically registers your services as commands, event subscribers, etc.
        bind:
            $environment: '%env(APP_ENV)%'
            $space: '%env(SPACE)%'
            $accessToken: '%env(MY_BIG_SECRET)%'

Now we can add a constructor to our controller that will let us access our secrets:

  private $space;

  private $accessToken;

  private $environment;

  public function __construct(string $environment, string $space, string $accessToken) {

    $this->environment = $environment;

    if ($environment === 'prod') {
      $this->space = getenv('CONTENTFUL_SPACE');
      $this->accessToken = getenv('CONTENTFUL_ACCESS_TOKEN');
    }
    else {
      $this->space = $space;
      $this->accessToken = $accessToken;
    }
  }

You might spot that we set the secrets differently depending on the environment. Of course, since we don't commit the .env.local file, that won't be available when we deploy. Instead, I want to manage my secrets via Elastic Beanstalk. We'll get to that in a minute.

Putting it on Elastic Beanstalk

AWS's instructions for putting a Symfony app onto EC2 via Elastic Beanstalk are pretty good. There's a couple of gotchas, mind you. The first, as covered in the documentation, is to make sure you set your document root to /public, either via the console or using a configuration file. The second is Apache Pack.

Apache Pack

Once you've installed Symfony, install Apache pack straight away:

composer require symfony/apache-pack

I spent a few hours banging my head against the wall tryting to figure out why only the front page would load and all of my routes threw 404s. Mostly, Apache pack puts an .htaccess file in your public directory so you can use it on a server. The AWS documentation rather glosses over this quite key detail.

It's also important to make sure that your instance is actually running Apache (again, in the Configuration screen in the Elastic Beanstalk console. My setup defaulted to Nginx, and again it took me a minute or two to figure out that Apache Pack wasn't working because of course it won't work with Nginx.

Let's get it deployed

You could manually upload your zipped-up code using the console. You could go a step further, and install the Elastic Beanstalk CLI locally, and deploy it like that. But I'm a big fan of a Bitbucket Pipeline, since that way you don't have to remember all the steps, and even if you ever do have to remember you can just read the pipeline file.

I wrestled with doing a full proper install of EBCLI on an Alpine image of my own, but in the end decided to take the line of least resistance and use the Bitbucket Elastic Beanstalk pipeline. You can see I've made some minor alterations, but really it works a treat the way it is:

image: atlassian/default-image:2

pipelines:
  branches:
    master:
      - step:
          name: Zip it up
          script:
            - zip -r application.zip .
          artifacts: 
            - application.zip 
      - step:
          name: Put it out
          script:
            - pipe: atlassian/aws-elasticbeanstalk-deploy:0.5.0
              variables:
                AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
                AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
                AWS_DEFAULT_REGION: "eu-west-2"
                APPLICATION_NAME: "sitemap-app"
                ENVIRONMENT_NAME: 'sitemap-app-dev'
                ZIP_FILE: "application.zip"

Those secret variables

Last but not least, you'll need to add the configuration we talked about earlier to Elastic Beanstalk. I've done it via the console - just go to your EB console, go to configuration and in the Environment Properties for CONTENTFUL_SPACE and CONTENTFUL_ACCESS_TOKEN, so that Symfony will be able to use them to construct the URL.

Linking it all up

Last of all, you're going to need to add a CloudFront behaviour to point your domain's sitemap.xml route to the Symfony application. You'll need a load balancer for the environment, which you can configure in the Elastic Beanstalk console (I don't think you can use a plain EC2 instance as a CloudFront origin). You can then add the load balancer as an origin in your site's CloudFront distribution, then add a new behaviour for /sitemap* to route requests to the Symfony sitemap.

And that's about all we've got time for, folks. If you've followed along, you should now be able to navigate to yoursite.com/sitemap.xml, and see a sitemap like this one.