Drupal News

Acro Media: Automated Drupal Code Testing: What, Why, and When To Do It

Main Drupal Feed - Tue, 04/10/2018 - 14:45

Automated testing is like flossing your teeth: you know you should do it, you might even tell people you do it, but chances are you don't do it nearly as often or as consistently as you ought to. Maybe you only run one automated test. On five percent of your code. Sometimes.

Manual testing vs automated testing

Manual testing—where a real live person clicks around and verifies that the code does everything it's supposed to do—has its uses. But for large-scale projects, or in cases where you need to test the same code repeatedly, automated testing can be both more cost-effective and more fruitful. You know how your eye can look at the same spelling error six times without seeing it? Automated testing can catch issues like that. It's great for rote, boring tasks that humans gloss over.

In test-driven development, you actually build the tests first and then write the code that fulfills those tests. But you don't necessarily have to do automated testing that way. Tests can be written afterwards. Sometimes it's old code that gets automated tests built for it, for instance.

Writing the test probably takes more time than it takes to initially test it. But then it's done, and you can re-run the test any time you make any change to that site. Even if you don't change anything near that code, you can run the test anyways and catch those instances where you're like, "I'm sure it won't break that"…but it does.

Drupal maintainers try to be really strict about the automated tests. Since lots of people use the modules, and lots of people contribute to them, you have all these different people submitting code. Having a good test suite can really improve your confidence in a module. If each submission comes with new tests that you can run and verify, you can be far more confident in the quality of that code.

Time savings

Say you do 10 hours of manual testing for each release. If, on the other hand, you spend 10 hours writing automated tests for each release, then for release #2, you're actually doing 20 hours of testing, and for release #3 you're doing 30 hours of testing. You're only putting in 10 man hours each time, but your test suite keeps growing, and the scope of what you're testing increases exponentially because you can rerun the same tests each time.

Why is automated testing such a hard sell?

Going back to the flossing example: why don't you floss? It takes 60 seconds. But you only really get the benefits if you do it all the time.

More to the point: not doing it takes time to become bad. Skipping automated testing on your first release is maybe not a big deal. Your code base is small, it's probably only doing a couple things, so manual testing is very feasible. But as the project grows and gets more complex, manual testing becomes increasingly unwieldy, and then you get to a point where you think, we're too far into this to add automated testing now.

But the truth is you can start at any point. It's never too late to start proactively doing things that will benefit your site.

What prevents people from getting started?

Usually the thing that stymies people is that they're not set up for automated testing: they don't have a continuous integration environment or a nice testing environment to run the tests on. Or maybe they've neglected it for so long that the regular tests don't work anymore; they write their first test and they have all these other problems because they've let things languish, so they give up.

What is continuous integration, you ask? It means every time you do a change and you push it out, it attempts to integrate it in with the whole project. It will deploy it to a server, it will compile the code if necessary, it will run code standards and automated tests and so on. When you have that stuff all set up to run automatically, you just make the code, push it to your version control, and forget about it. If something goes south, it'll send you an email saying this didn't pass. Otherwise, you don't have to think about it again.

How much of your code should be covered by automated testing?

You kind of want to be in that 95% range, although it's true that you can have parts that aren't easily testable. You can cover a lot of code, but you might still be missing use cases. Maybe you test taxes, and you test discounts, but you don't test taxes and discounts together. So technically you have full code coverage, but you're not using them in combination. So code coverage is a nice metric, but it doesn't tell the whole story.

TLDR: Automated testing is like flossing. You should really do it.

More from Acro Media Chat with us

If you'd like to talk about adding automated testing to your Drupal Commerce website, or if you just want to to discuss how Drupal Commerce fits into your ecommerce solution, give us a shout. We're happy to chat.

roomify.us: Tutorial: showing BEE reservations on an event calendar

Main Drupal Feed - Tue, 04/10/2018 - 13:55
BEE makes it easy to quickly implement all kinds of booking & reservation use cases. We've created a new video that walks you through setting up an event calendar displaying BEE reservations.

ComputerMinds.co.uk: Rendering Drupal 8 fields (the right way)

Main Drupal Feed - Tue, 04/10/2018 - 11:21

Once upon a time, we wrote an article about how to render fields on their own in Drupal 7, which was really handy because Drupal 7 wasn't always intuitive. It's common to want to display a field outside of the context of its main entity page, like showing author information in a sidebar block or in a panel, but you had to just know which functions to use. Drupal 8 has come along since then using 'grown up' things like objects and methods, which actually makes the job a little easier. So now we have this:

The short answer $node->field_my_thing->view();Quick reference

I'll cover these in detail below, but here are the things you might want to be doing:

  1. Render a field as if it were on the node/entity page (e.g. with the markup from the normal formatter and full field template)

Drupal 7: field_view_field(), optionally passing a view mode string, or formatter settings array.
Drupal 8: $node->field_my_thing->view(), optionally passing a view mode string, or formatter settings array.

  1. Just get a single value out of a field (i.e. raw values, usually as originally inputted)

Drupal 7: field_get_items() and then retrieve the index you want from the array.
Drupal 8: $node->field_my_thing->get(0)->value, passing just the index you want to get(). Properties other than 'value' may be available.

  1. Render a single value as if it were on the node/entity page (e.g. with the normal formatter's markup, but not all the wrappers that Drupal's field templates give you)

Drupal 7: field_view_value(), optionally passing a view mode string, or formatter settings array, but always passing the actual items array.
Drupal 8: $node->field_my_thing->get(0)->view(), passing just the index you want to get() and optionally passing a view mode string, or formatter settings array to view().

The long answer

Now that entities like nodes are properly classed objects, and fields use the fancy new Typed Data API, we don't need to care about the underlying data structure for nodes or their fields, we can just call the method to perform the operation we want! You know, just what methods are supposed to be for! You want to view a field? Just call its 'view' method.

The output will be automatically sanitised and goes through the normal formatter for the field, as well as the regular field template. You can specify whether you want it rendered as if it were in a teaser or other view mode, by passing in the view mode string, just as we did with field_view_field(). (Or you might have used something like $node->field_my_thing['und'][0]['value'] - in which case, go back and read our article for Drupal 7!)


Or even override the formatter to be used altogether (which is handy if the field would normally be hidden in any view mode):

$node->field_my_thing->view([ 'type' => 'image', 'label' => 'hidden', 'settings' => array( 'image_style' => 'larger_thumbnail', 'image_link' => 'content', ), ]);

This does assume that your field ('field_my_thing') in my examples does at least exist on your node (even if it's empty). You may want to wrap the whole code in a try/catch block, just in case it might not.

For bonus points, you could load up the normal formatter settings, and tweak them:

use Drupal\Core\Entity\Entity\EntityViewDisplay; // If you have the entity/node you want, use collectRenderDisplay() as it may // already be statically cached, plus it goes through various alter hooks. $display_options = EntityViewDisplay::collectRenderDisplay($node, 'teaser') ->getComponent('field_images'); // Otherwise - or if you intentionally want to re-use the settings from another // unrelated entity type, bundle or display mode - just load the display config. $display_options = EntityViewDisplay::load('pagaraph.media_pod.teaser') ->getComponent('field_images'); // Then tweak those display options and view the field. $display_options['settings']['image_style'] = 'even_bigger_thumbnail'; $node->field_my_thing->view($display_options);

This all assumes you've at least loaded your node, or other entity. (It works with any content entity, like terms, paragraphs, etc!) You'd probably be putting the result of the view() method (which will be a render array) into a variable to be used in a twig template via a preprocess hook. Or maybe you're just adding it into an existing render array, like a custom block plugin. (Either way, you probably shouldn't then be rendering it into a string yourself, let Drupal do that for you.)

You can even just view a single item within a multi-value field like this, here using an 'if' condition to be a bit more graceful than a try/catch. This is the equivalent of using field_view_value() from Drupal 7, so also skips Drupal's full field template, though includes markup produced by the field's formatter:

// View the third value, as long as there is one. if ($third = $node->field_my_thing->get(2)) { $output = $third->view(); } else { $output = []; }

That helps you see how you might get a single value too, with the get() method, though note that it still returns a classed object. To just get a raw value, without any wrapping markup or value processing/sanitisation, you might want to use something like this, instead of Drupal 7's field_get_items() :

$text = $node->field_my_thing->get(0)->value; // If the field is just a single-value field, you can omit the get() part. $value = $node->field_single_thing->value; // Some types of field use different properties. $url = $node->field_my_link->uri; // You can use getValue() to get all the properties (e.g. text value + format). $text_values = $node->field_formatted_text->getValue(); // References can be chained! $referenced_title = $node->field_my_reference->entity->field_other_thing->value;

In Drupal 7, there was also confusion around what to do for multilingual content. In Drupal 8, as long as you've got the translation you want first, all the methods I've discussed above will get you the appropriate values for your language. To get a specific translation, use:

if ($node->hasTranslation($candidate)) { $node = $node->getTranslation($langcode); } This Rocks.

You get to use proper modern methods on a proper typed data API. Sanitisation is done for you. You don't need to care what the underlying data structure is. And you don't need to remember some magic global procedural functions, because the methods are obvious, right there on the thing you want to use them on (the field item class). If the Drupal 7 version of this was brilliant, that makes the Drupal 8 version of this even better. Brilliant-er?

OPTASY: What Are the Differences Between PHPStorm and WebStorm? Which IDE Is Right for You?

Main Drupal Feed - Tue, 04/10/2018 - 09:38
What Are the Differences Between PHPStorm and WebStorm? Which IDE Is Right for You? radu.simileanu Tue, 04/10/2018 - 09:38

Feeling stuck? Can't seem to put a finger on at least a few clear differences between PHPStorm and WebStorm? And you need to choose the most suitable IDE software for web development?

There sure must be some strong differences, other than:

PHPStorm doesn't provide JavaScript-oriented plugin support right out-of-the-box like WebStorm does.

Now, before we go “hunting” some key differences between PHPStorm and WebStorm, I'd like to add one last recommendation to consider when you select the right IDE for you:

It all comes down to evaluating various solutions and identifying not THE BEST, but the application's perfectly suited to your specific needs.

That being said, without further ado, let's evaluate the “candidates”!

Jeff Geerling's Blog: A modern way to build and develop Drupal 8 sites, using Composer

Main Drupal Feed - Tue, 04/10/2018 - 03:27

The Drupal community has been on an interesting journey since the launch of Drupal 8 in 2015. In the past three years, as the community has started to get its sea legs 'off the island' (using tools, libraries, and techniques used widely in the general PHP community), there have been growing pains.

One area where the pains have been (and sometimes still are) highly visible is in how Drupal and Composer work together. I've written posts like Composer and Drupal are still strange bedfellows in the past, and while in some ways that's still the case, we as a community are getting closer and closer to a nirvana with modern Drupal site building and project management.

For example, in preparing a hands-on portion of my and Matthew Grasmick's upcoming DrupalCon Nashville lab session on Composer and Drupal, I found that we're already to the point where you can go from literally zero to a fully functional and complete Drupal site codebase—along with a functional local development environment—in about 10 or 15 minutes:

Agiledrop.com Blog: AGILEDROP: Day zero at DrupalCon Nashville

Main Drupal Feed - Tue, 04/10/2018 - 00:22
It was a long trip from Slovenia to Nashville, but after a good night sleep, I headed towards Music City Center for the first day of DrupalCon. Mondays at DrupalCon are reserved for sub-events called summits. There were Decoupled Summit, Higher Education Summit, Government Summit, Media and Publishing Summit, Nonprofit Summit, Community Summit and Business Summit. The last is the one I picked to attend.    Business Summit The event was organized by Elia Albarran, Four Kitchens and Aimee Degnan, Hook 42. The idea of the event is to facilitate open discussions between Drupal agency owners… READ MORE

Nuvole: Config Distro

Main Drupal Feed - Mon, 04/09/2018 - 16:06
A new workflow for distribution configuration management.Drupal core workflow limitations

It is not a secret that the configuration management in Drupal 8 core was made for only one specific use case: move configuration from one environment to another for the same site. Everything else was left for contrib to find solutions for. Config Installer is necessary to set up a site starting from existing configuration and Config Split is needed to have environment specific configuration that go beyond simple configuration overrides, like development modules enabled only locally. But the whole workflow still assumes that developers - most often in the same team - have control over the environments and deployment of the site.

Distributions are not covered

Distributions are very different. When maintaining a distribution one doesn't develop a specific site but rather the template to build different sites with. Deploying configuration as we do it for a single site does not have any meaning because there is no one production site to deploy to. Each site based off a distribution is its own site with its own site uuid and its own possible customisations and possibly its own development and production environments. At the same time as a consumer of a distribution - a site owner of a site based off a distribution - one wants to update the distributions features while keeping the customisations.

Update hooks?

Different distributions handle this in different ways. A first approach to solve this issue is treating it the same way you would a schema update and change the configuration on a site with an update hook. The problem with this is that update hooks are meant to fix the database to be able to run with the version of code that is in use on the site. The first thing one needs to do after updating the code base is to run the update hooks to realign the database. Configuration management has different needs and when updating the configuration in an update hooks means that one has to be careful to check that the configuration is in the expected state. And the only recourse is to log and make sure the user is informed what manual actions he needs to do to update the configuration himself. As there is no place to involve the user with a choice while update hooks are run.

Config Distro

We propose a new kind of workflow, in essence it allows developers of a distribution to say how the "distributions configuration" is supposed to be updated and it allows site owners of a site based on a distribution to treat the distribution maintainers as a developer on their team. At the same time this does not interfere with the configuration management workflow for staging and deploying configuration to the production site.

The primary function of Config Distro is to allow updating a distributions configuration in a new workflow and give the site owners more control and insight on how the configuration of their site changes. It is a new way to imagine the configuration management system in a way that lets sites own their configuration but allowing distributions to provide updated configuration for existing sites.

A new dimension

In a blog post one year ago I talked about different dimensions of configuration management;

  • vertical: moving configuration between environments but for the same site
  • horizontal: moving configuration between different sites. for example for distribution updates.

Config Distro is a UI and a drush command for that workflow. It is intended to be used by site builders who base their site on a distribution and want to import configuration updates from the updated distribution. The UI is essentially a parallel to configuration import screen for the workflow supported by core. But instead of importing the configuration from the files used to move/deploy configuration between environments you import from a special configuration storage which starts out with the sites active configuration and has the distributions updates applied to it. This means you do not have problems with mismatched site uuids or things being removed that you added and were not part of the distribution. And updates can apply (or not) to translations even if the distribution did not have languages.

Where do updates come from

If you only install Config Distro, the import screen eternally shows that there is nothing to import. This is because the module only provides the framework for this to work, ie the UI and the drush command. The module is not opinionated on how or what should be updated. All the complexity can be addressed by a distribution maintainer with the help of a ConfigFilter plugin (the same way Config Split and Config Ignore work). One such module is Config Sync. All the complexity of finding out what what configuration a module has originally shipped with, what it does now and whether a user has changed the originally installed configuration is left to Config Sync and its dependencies.


Just like Config Ignore allows you to opt out of some of the configuration management workflows, Config Distro has a Config Distro Ignore module that lets you retain certain configuration from being changed when you hit the "import" button. The "Retain configuration" button is available right next to the "View differences" button.

Clicking it leads to a form that lets you choose to retain the configuration permanently or only for this specific update. It also allows you to ignore a update for a specific language.


In our team we set up a site based on a distribution. We added our own login module for our single sign on and added a few webforms. Now there is a new version of the distribution with some new features and I would like upgrade the site to using the new features. I am tasked with updating the site, here is what I do:

  • I update the code of the distribution by specifying the new version in composer.json and do a composer update to get the updated code*
  • I run the database updates drush updb to align the database with the code*
  • I go to the distro update screen provided by Config Distro
    • This screen looks very familar, it looks the same way as when I import configuration changes my team mates made after I get their code with git pull.
    • I see a new module is going to be installed and a few views and settings from the distribution are updated.
    • Our webforms are not going to be removed and our custom modules not uninstalled.
  • I click "import all"
  • Now my site has the updated code and configuration so I export the configuration for deployment: drush cex and commit*
  • My colleges get the update the same way they get normal changes that happen during development*:
    • git pull get the code
    • composer install get the external code (vendor, contrib, etc)
    • drush updb align the database with the new code base.
    • drush cim import the sync configuration.

* This is the same procedure as for any module update on any site.
Distributions may add a "Auto upgrade" setting and then import the Config Distro in a post_update_hook to bypass the manual step required of site administrators upgrading the distribution.


Config Distro provides a new workflow for updating distributions with a familiar feel. It recognizes that update hooks are not an adequate solution for updating a sites configuration when a site owns the configuration and site administrators may have changed the initially provided configuration.
It allows developers of distributions to alter the sites configuration through ConfigFilter plugins and it gives site administrators a choice of what to import.
Config Distro is just the framework for extending cores configuration management to allow managing configuration changes to get into the site from a third party such as the distribution maintainers. It does not interfere with the traditional workflow and importing the configuration updates from a distribution should be seen as any other configuration change such as adding a new view or changing permissions: You go to an admin page and you click some things in a form, then you export the configuration and deploy it.

Future - CMI 2.0

While it already works, Config Distro is still in an alpha state. Config Distro is part of a larger effort to improve the configuration management in Drupal. You can find further information and participate in the discussion on drupal.org

Tags: Drupal 8Code Driven DevelopmentConfiguration ManagementDrupal Planet

ThinkDrop Consulting: Announcing Provision 4: Leaving the Aegir Nest

Main Drupal Feed - Mon, 04/09/2018 - 15:27
Announcing Provision 4: Leaving the Aegir Nest Jon Pugh Mon, 04/09/2018 - 11:27

As we head into DrupalCon week I've got something big to announce. With the blessing of the Aegir core maintainers, I am taking the 4.x branch of Provision I have been working on and I am separating it from the Aegir Project.

Provision 4 will still power Aegir. We are working on a patch to Hostmaster that will allow us to run a different command other than Drush, allowing Provision4 to become the primary back-end to Aegir 4. This means it will also be able to power the current generation DevShop.

Hook 42: February Accessibility (A11Y) Talks

Main Drupal Feed - Mon, 04/09/2018 - 11:47

In February, we welcomed Scott Vinkle as our guest speaker. Scott is an accessibility expert and front-end developer for Shopify who spoke to our meetup group about creating accessible React JS apps. Scott has been active in the accessibility community since 2011. He has worked with many groups including the accessibility project, the a11y tour, CodePen Ottawa, and many, many others.

Roman Agabekov: How we applied SA-CORE-2018-002 for several dozen Drupal-powered sites

Main Drupal Feed - Mon, 04/09/2018 - 09:41
How we applied SA-CORE-2018-002 for several dozen Drupal-powered sites Submitted by admin on Mon, 04/09/2018 - 09:41

Here is a brief account of how we applied the most critical Drupal security update in the past couple of years to web projects we support and monitor.


Lucius Digital: 26 Cool Drupal modules for site builders | April 2018

Main Drupal Feed - Mon, 04/09/2018 - 08:52
The biggest ‘module’ being updated was the Drupal core of course: Drupal 8.5.0 was released about a month ago, read more. Also, everything else what stood out to the module updates of last month, enjoy:

Jeff Geerling's Blog: Installing PHP 7 and Composer on Windows 10

Main Drupal Feed - Mon, 04/09/2018 - 03:23

I am working a lot on Composer-based Drupal projects lately (especially gearing up for DrupalCon Nashville and my joint workshop on Drupal and Composer with Matthew Grasmick), and have been trying to come up with the simplest solutions that work across macOS, Linux, and Windows. For macOS and Linux, getting PHP and Composer installed is fairly quick and easy. However, on Windows there seem to crop up little issues here and there.

Since I finally spent a little time getting the official version of PHP for native Windows installed, I figured I'd document the process here. Note that many parts of this process were learned from the concise article Install PHP7 and Composer on Windows 10 from the website KIZU 514.

Install PHP 7 on Windows 10

Love Huria: Cool things you can do with Sass - Part 1

Main Drupal Feed - Mon, 04/09/2018 - 00:00

I have been using Sass for like past two years and now I’m a huge fan. Even though we were doing pretty much alright with writing CSS but it never gave us that kind of flexibility that Sass provides like one of the things could be managing the complexity in stylesheets as our apps get more and more substantial. Anyways, Enough about my experience already as today we have got a bunch of cool things to cover!

What is Sass?

It’s a CSS preprocessor, that’s what you will get if you start googling and its true but hold that...

Wim Leers: API-First Drupal: file uploads — 572 comments summarized

Main Drupal Feed - Sun, 04/08/2018 - 21:11

This blog post summarizes the 572 comments spanning 5 years and 2 months to get REST file upload support in #1927648 committed. Many thanks to everyone who contributed!

From February 2013 until the end of March 2017, issue #1927648 mostly … lingered. On April 3 of 2017, damiankloip posted an initial patch for an approach he’d been working on for a while, thanks to Acquia (my employer) sponsoring his time. Exactly one year later his work is committed to Drupal core. Shaped by the input of dozens of people! Just *look at that commit message!*

Background: API-First Drupal: file uploads!.

  • Little happened between February 2013 (opening of issue) and November 2015 (shipping of Drupal 8).
  • Between February 2013 and April 2014, only half a dozen comments were posted, until moshe weitzman aptly said Still a gaping hole in our REST support. Come on Internets ….
  • The first proof-of-concept patch followed in August 2014 by juampynr, but was still very rough. A fair amount of iteration occurred that month, between juampynr and Arla. It used base64 encoding, which means it needed 33% more bytes on the wire to transfer a file than if it were transmitted in binary rather than base64.
  • Then again a period of silence. Remember that this was around the time when we were trying to get Drupal 8 to a shippable state: the #1 priority was to stabilize, fix critical bugs. Not to add missing features, no matter how important. To the best of my knowledge, the funding for those who originally worked on Drupal 8’s REST API had also dried up.
  • In May 2015, another flurry of activity occurred, this time fueled by marthinal. Comment #100 was posted. Note that all patches up until this point had zero validation logic! Which of course was a massive security risk. marthinal is the first to state that this is really necessary, and does a first iteration of that.
  • A few months of silence, and then again progress in September, around DrupalCon Barcelona 2015. dawehner remarked in a review on the lack of tests for the validation logic.
  • In February 2016 I pointed out that I’m missing integration tests that prove the patch actually works. To which Berdir responded that we’d first need to figure out how to deal with File entity type access control!
  • Meanwhile, marthinal works on the integration test coverage in 2016. And … we reached comment #200.
  • In May 2016, I did a deep review, and found many problems. Quick iterations fix those problems! But then damiankloip pointed out that despite the issue being about the general File (de)serialization problem, it actually only worked for the HAL normalization. We also ended up realizing that the issue so far was about stand-alone File entity creation, even though those entities cannot be viewed stand-alone nor can they be created stand-alone through the existing Drupal UI: they can only be created to be referenced from file fields. And consequently, we have no access control logic for this yet, nor is it clear how access control should work; nor is it how validation should work! Berdir explained this well in comment 232. This lead us to explore moving parts of https://www.drupal.org/project/file_entity into core (which would be a hard blocker). The issue then went quiet again.
  • In July 2016, garphy pointed out that large file uploads still were not yet supported. Some work around that happened. In September, kylebrowning stressed this again, and provided a more detailed rationale.
  • Then … silence. Until damiankloip posted comment #281 on April 3, 2017. Acquia was sponsoring him to work on this issue. Damian is the maintainer of the serialization.module component and therefore of course wanted to see this issue get fixed. My employer Acquia agreed with my proposal to sponsor Damian to work on REST file upload support. Because after 280 comments, some fundamental capabilities are still absent: this was such a complex issue, with so many concerns and needs to balance, that it was nigh impossible to finish it without dedicated time.
    To get this going, I asked Damian to look at the documentation for a bunch of well-known sites to observe how they handle file uploads. I also asked him to read the entire issue. Combined, this should give him a good mental map of how to approach this.
  • #281 was a PoC patch that only barely worked but did support binary (non-base64) uploads. damiankloip articulated the essential things yet to be figured out: validation and access checking. Berdir chimes in with his perspective on that in #291 … in which he basically outlines what ended up in core! Besides Berdir, dagmar, dawehner, garphy, dabito, ibustos all chimed in and influenced the patch. Berdir, damiankloip and I had a meeting about how to deal with validation, and I disagreed with with both of them. And turned out to be very wrong! More feedback is provided by the now familiar names, and the intense progress/activity continues for two months, until comment #376!
  • Damian got stuck on test coverage — and since I’d written most of the REST test coverage in the preceding months, it made sense for me to pick up the baton from Damian. So I did that in July 2017, just making trivial changes that were hard to figure out. Damian then continued again, expanding test coverage and finding a core bug in the process! And so comment #400 was reached!
  • At the beginning of August, the patch was looking pretty good, so I did an architectural review. For the first time, we realized that we first needed to fix the normalization of File entities before this could land. And many more edge cases need to be tested for us to be confident that there were no security vulnerabilities. blainelang did manual testing and posted super helpful feedback based on his experience. Blaine and Damian tag-teamed for a good while, then graphy chimed in again, and we entered September. Then dawehner chimed in once more, followed by tedbow.
  • On September 6 2017, in comment #452 I marked the issue postponed on two other issues, stating that it otherwise looked tantalizingly close to RTBC. aheimlich found a problem nobody else had spotted yet, which Damian fixed.
  • Silence while the other issues get fixed … and December 21 2017 (comment #476), it finally was unblocked! Lots of detailed reviews by tedbow, gabesullice, Berdir and myself followed, as well as rerolls to address them, until I finally RTBC‘d it … in comment #502 on February 1 2018.
  • Due to the pending Drupal 8.5 release, the issue mostly sat waiting in RTBC for about two months … and then got committed on April 3 2018!!!

Damian’s first comment (preceded by many hours of research) was on April 3, 2017. Exactly one year later his work is committed to Drupal core. Shaped by the input of dozens of people! Just look at that commit message!

  • API
  • Acquia
  • Drupal

Wim Leers: API-First Drupal: file uploads!

Main Drupal Feed - Sun, 04/08/2018 - 21:09

Drupal 8’s REST API has been maturing steadily since the Drupal 8.0.0 was released in November 2015. One of the big missing features has been file upload support. As of April 3 2018, Drupal 8.6 will support it, when it ships in September 2018! See the change record for the practical consequences: https://www.drupal.org/node/2941420.

It doesn’t make sense for me to repeat what is already written in that change record: that already has both a tl;dr and a practical example.

What I’m going to do instead, is give you a high-level overview of what it took to get to this point: why it took so long, which considerations went into it, why this particular approach was chosen. You could read the entire issue (#1927648), but … it’s one of the longest issues in Drupal history, at 572 comments1. You would probably need at least an entire workday to read it all! It’s also one of the longest commit messages ever, thanks to the many, many people who shaped it over the years:

Issue #1927648 by damiankloip, Wim Leers, marthinal, tedbow, Arla, alexpott, juampynr, garphy, bc, ibustos, eiriksm, larowlan, dawehner, gcardinal, vivekvpandya, kylebrowning, Sam152, neclimdul, pnagornyak, drnikki, gaurav.goyal, queenvictoria, kim.pepper, Berdir, clemens.tolboom, blainelang, moshe weitzman, linclark, webchick, Dave Reid, dabito, skyredwang, klausi, dagmar, gabesullice, pwolanin, amateescu, slashrsm, andypost, catch, aheimlich: Allow creation of file entities from binary data via REST requests

Thanks to all of you in that commit message!

I hope it can serve as a reference not just for people interested in Drupal, but also for people outside the Drupal community: there is no One Best Practice Way to handle file uploads for RESTful APIs. There is a surprising spectrum of approaches2. Some even avoid this problem space even entirely, by only allowing to “upload” files by sending a publicly accessible URL to the file. Read on if you’re interested. Otherwise, go and give it a try!

Design rationale


  • Request with Content-Type: application/octet-stream aka “raw binary” as its body, because base64-encoded means 33% more bytes, implying both slower uploads and more memory consumption. Uploading videos (often hundreds of megabytes or even gigabytes) is not really feasible with base64 encoding.
  • Request header Content-Disposition: file; filename="cat.jpg" to name the uploaded file. See the Mozilla docs. This also implies you can only upload one file per request. But of course, a client can issue multiple file upload requests in parallel, to achieve concurrent/batch uploading.
  • The two points above mean we reuse as much as possible from existing HTTP infrastructure.
  • Of course it does not make sense to have a Content-Type: application/octet-stream as the response. Usually, the response is of the same MIME type as the request. File uploads are the sensible exception.
  • This is meant for the raw file upload only; any metadata (for example: source or licensing) cannot be associated in this request: all you can provide is the name and the data for the file. To associate metadata, a second request to “upgrade” the raw file into something richer would be necessary. The performance benefit mentioned above more than makes up for the RTT of a second request in almost all cases.


  • php://input because otherwise limited by the PHP memory limit.


  • In the case of Drupal, we know that it always represents files as File entities. They don’t contain metadata (fields), at least not with just Drupal core; it’s the file fields (@FieldType=file or @FieldType=image) that contain the metadata (because the same image may need different captions depending on its use, for example).
  • When a file is uploaded for a field on a bundle on an entity type, a File entity is created with status=false. The response contains the serialized File entity.
  • You then need a second request to make the referencing entity “use” the File entity, which will cause the File entity to get status=true.
  • Validation: Drupal core only has the infrastructure in place to use files in the context of an entity type/bundle’s file field (or derivatives thereof, such as image fields). This is why files can only be uploaded by specifying an entity type ID, bundle ID and field name: that’s the level where we have settings and validation logic in place. While not ideal, it’s pragmatic: first allowing generic file uploads would be a big undertaking and somewhat of a security nightmare.
  • Access control is similar: you need create access for the referencing entity type and field edit access for the file field.

If we combine all these choices, then we end up with a new file_upload @RestResource plugin, which enables clients to upload a file:

  1. by POSTing the file’s contents
  2. to the path /file/upload/{entity_type_id}/{bundle}/{field_name}, which means that we’re uploading a file to be used by the file field of the specified entity type+bundle, and the settings/constraints of that field will be respected.
  3. … don’t forget to include a ?_format URL query argument, this determines what format the response will be in
  4. sending file data as a application/octet-stream binary data stream, that means with a Content-Type: application/octet-stream request header. (This allows uploads of an arbitrary size, including uploads larger than the PHP memory limit.)
  5. and finally, naming the file using the Content-Disposition: file; filename="filename.jpg" header
  6. the five preceding steps result in a successfully uploaded file with status=false — all that remains is to perform a second request to actually start using the file in the referencing entity!
Four years in the making — summarizing 572 comments

From February 2013 until the end of March 2017, issue #1927648 mostly … lingered. On April 3 of 2017, damiankloip posted an initial patch for an approach he’d been working on for a while, thanks to Acquia (my employer) sponsoring his time. Exactly one year later his work is committed to Drupal core. Shaped by the input of dozens of people! Just look at that commit message!

Want to actually read a summary of those 572 comments? I got you covered!

  1. It currently is the fifth longest Drupal core issue of all time! The first page, with ~300 comments, is >1 MB of HTML. ↩︎

  2. Examples: Contentful, Twitter, Dropbox and others↩︎

  • API
  • Acquia
  • Drupal

Another Drop in the Drupal Sea: Migrating from Drupal 6 to Drupal 8

Main Drupal Feed - Sun, 04/08/2018 - 16:31
Migrating from Drupal 6 to Drupal 8 Marc Sun, 04/08/2018 - 11:31am

Well, I've finally done it! I migrated this blog from Drupal 6 to Drupal 8. I did a test run yesterday with a personal blog of mine and found the process was relatively easy. Both sites are relatively simple.

There are other blog posts about the process as well as documentation on Drupal.org so I won't repeat lots of details.

I'm running Drupal 8.5.1 as of this blog post. I chose to use all of the various migration modules that come with Drupal 8 core, including the two marked as experimental. Once I had them enabled I clicked the link to get to the upgrade form in the UI. One of the sites did have file uploads and all of them were pulled in seamlessly. I had created a backup of the sites/[site-name] directory and placed it in my new Drupal 8 sites directory.

Here are issues I encountered:

  • Administration Menu is (apparently) not properly ported to Drupal 8 and it blew things up on the site yesterday. I had to manually clean things up in the database so that module was not enabled in my Drupal 8 site and clear cache.
  • The taxonomy term reference to the Tags vocabulary needed the field setting updated so that it selected the Tags vocabulary.
  • When I click the user in the Toolbar it does not switch the tray so that it shows View profile, Edit profile and Log out. (That's still an issue as I write this. I haven't investigated it enough to figure out what's going wrong, nor have I filed an issue.)
  • The feed that taxonomy provides for terms has changed slightly. I filed an issue to get Planet Drupal to use the new feed.
  • No views were imported.
  • The pathauto patterns weren't imported.
  • Disqus doesn't handle the migration.
  • Other than those things, I can't say I ran into much of anything else. And, aside from the site blowing up from Administration Menu, there wasn't much that presented a real challenge.

If you are reading this post on Planet Drupal, then you know I'm back up and running!

I'll still need to theme this site again. And I'll need to replace some functionality that was previously provided by Advanced Blog. I haven't yet installed and tested out the Drupal 8 port of Tagadelic.

One more note: I decided to just delete the comments that I had for the Drupal 6 version of this site since I don't want to use the Comment module, preferring to use Disqus instead.


Drupal core announcements: Core topic discussions at DrupalCon Nashville

Main Drupal Feed - Sat, 04/07/2018 - 15:51

DrupalCon Nashville includes a full track of core conversations where you can learn about current topics in Drupal core development, and a week of sprints where you can participate in shaping Drupal's future.

In addition to the core conversations, we have a few meetings on specific topics for future core development. These meetings will be very focused, so contact the listed organizer for each if you are interested in participating. There are also birds-of-a-feather (BoF) sessions, which are open to all attendees without notice.

Also be sure to watch Dries' keynote for ideas about Drupal's future! Check out the extended Dries Q&A session on Thursday as well to get even more questions answered.

Time Topic Organizer Monday, 9 April, 10:00 Configuration validation to support REST and JS Wim Leers Tuesday, 10 April, 10:45 Improving Drupal's evaluator experience (BoF) tedbow Tuesday, 10 April, 15:45 Layout Initiative meeting tim.plunkett Wednesday, 11 April, 10:45 Official local development environment (BoF) tedbow Wednesday, 11 April, 14:15 Media roadmap meeting phenaproxima Friday, 13 April, 09:00 Release cycle changes discussion (only core committers) Gábor Hojtsy Friday, 13 April, 11:00 Automated security updates hestenet

Commerce Guys: Visit the Commerce Saloon at DrupalCon Nashville

Main Drupal Feed - Sat, 04/07/2018 - 06:30

Commerce Guys is joining forces with some of our Technology Partners and several contributing agencies to promote Drupal Commerce at DrupalCon Nashville from April 10-12, 2018.

We are colocating our booths to create the Commerce Saloon, your one stop shop to learn all things Drupal Commerce. Our booths will feature jam band instruments, multiple demos (including a new store theme), exclusive swag, and case studies to help you learn how teams are succeeding with Drupal Commerce.

Come try Drupal Commerce 2.x

DrupalCon Nashville is the perfect time to learn what's new by joining our week long sprint at the "Power Up" tables by the Commerce Saloon. We'll be training new contributors and working on the project together using sprint kits powered by DRUD's ddev local development environment.

We prepared the following sessions to help you learn more about Drupal Commerce and its ecosystem:

  • Contributing to Drupal Commerce (for beginners)
    Tuesday, April 10th, 12:00 PM | Commerce Saloon: "Power Up" Table | By: Matt Glaman
  • Drupal Commerce 2.x Update and Roadmap Planning (add it to your conference schedule)
    Tuesday, April 10th, 3:45 PM | Room: 203A | By: Ryan Szrama / Bojan Zivanovic
  • Marketing and Selling the Drupal Commerce Ecosystem (as seen at DrupalCon Vienna)
    Wednesday, April 11th, 10:45 AM | Commerce Saloon: "Power Up" Table | By: Ryan Szrama
  • Decoupled Drupal Commerce / REST APIs (for developers)
    Wednesday, April 11th, 3:45 PM | Commerce Saloon: "Power Up" Table | By: Matt Glaman
  • Subscriptions and Recurring Billing in Commerce 2.x
    Thursday, April 12th, 10:45 AM | Commerce Saloon: "Power Up" Table | By: Bojan Zivanovic

Hear from every Commerce Saloon sponsor

There's a lot to be said about how Drupal Commerce is making merchant and agency teams more productive, and you don't just have to take our word for it. Each Commerce Saloon sponsor has something unique to teach you about succeeding in eCommerce, and we encourage you to seek them and their sessions out:

  • Acro Media (Booth 803) - Test drive Commerce POS at their booth and hear its business case from Becky and Josh! You can also purchase (for free) a limited edition Drupal Commerce t-shirt through Acro Media's demo site.
  • Authorize.Net (Booth 911) - Authorize.Net offers several payment tools that let merchants get paid securely online. We've joined forces to demo Accept.js, their new drop-in solution for PCI compliant payment.
  • Bluespark (Booth 908) - Bluespark contributed significantly to Commerce 2.x development via their Sport Obermeyer project (check out their awesome case study) and have long promoted Drupal Commerce as a hotel booking solution.
  • Commerce Guys (Booth 809) - Stop by for a demo of Belgrade, our new default store theme for Commerce 2.x, or for a demo of, Lean Commerce Reports, our first SaaS product that offers a plug-n-play sales dashboard for Drupal Commerce.
  • Drupal Commerce Technology Partners (Both 811) - This booth features representatives and demos from Avalara and Lockr. Talk to them about tax automation and about eCommerce security respectively.
  • MailChimp (Booth 813) - MailChimp has revitalized their approach to eCommerce email marketing and has a full integration available for Drupal in the MailChimp eCommerce module. Stop by to learn more!
  • Zivtech (Booth 909) - Zivtech has a long history of implementing eCommerce in Drupal, including joining the Drupal Commerce project in late 2009. Talk to them about using Drupal Commerce as a front-end for third party applications.

Finally, be sure to catch Promet Source's showcase session on helping The Corning Museum of Glass migrate from Commerce 1.x to Commerce 2.x and Rick Manelius's session on the dos and don'ts Drupal Commerce project estimation.

Schedule Time to Meet

If you're heading to DrupalCon, we'd love to chat about Drupal Commerce with you. Use our meeting request form to get on our calendar to discuss a particular project or need, or subscribe to our newsletter to be kept in the loop more generally.

Dcycle: Fast-track local Drupal 8 core patch development and testing

Main Drupal Feed - Sat, 04/07/2018 - 00:00

The process documented process for setting up a local environment and running tests locally is, in my opinion, so complex that it can be a barrier to even determined developers.

For those wishing to locally test and develop core patches, I think it is possible to automate the process down to a few steps and few minutes; here is an example with a core issue, #2273889 Don’t use one language’s plural index formula with another language’s string in the case of untranslated strings using format_plural(), which, at the time of this writing, results in the number 0 being displayed as 1 in certain cases.

Is it possible to start useful local development on this within 10 minutes on a computer with nothing installed except Docker? Let’s try…

Step 1: install Docker

Install and launch Docker. Everything we need, Apache web server, MySql server, Drush, Drupal, will reside on Docker containers, so we won’t need to install anything locally except Docker.

Step 2: launch a dev environment

I have create a project hosted on GitHub which will help you set up everything you need in Docker contains without local dependencies other than Docker, or any manual steps. Set it up by running:

git clone https://github.com/dcycle/drupal8_core_dev_helper.git && \ cd drupal8_core_dev_helper && \ ./scripts/deploy.sh`

This will create everything you need: a webserver container and database container, and your Drupal core code which will be placed in ./drupal8_core_dev_helper/drupal; near the end of the output of ./scripts/deploy.sh, you will see a login link to your development environment. Confirm you can access that local development environment at an address like (The port is random.)

The first time you run this, it will have to download Docker images with Drupal, MySQL, and install everything you need for local development. Future runs will be a lot faster.

See the project’s README for more details.

In your dev environment, you can confirm that the problem exists (provided the issue has not yet been fixed) by following the instructions in the “To reproduce this problem:” section of the issue description on your local development environment.

Any calls to drush can be run on the Docker container like so:

docker-compose exec drupal /bin/bash -c 'drush ...'

For example:

docker-compose exec drupal /bin/bash -c 'drush en locale language -y'

If you want to run drush directly, you can connect to your container like so:

docker-compose exec drupal /bin/bash

This will result in the following prompt on the container:


Now you can run drush commands directly on the container:

drush eval "print_r(\Drupal::translation()->formatPlural(0, '1 whatever', '@count whatevers', array(), array('langcode' => 'fr')) . PHP_EOL);"

Because the drupal8_core_dev_helper project also pre-installs devel on your environment, you can also confirm the problem exists by visiting /devel/php and executing:

dpm((string) (\Drupal::translation()->formatPlural(0, '1 whatever', '@count whatevers', array(), array('langcode' => 'fr'))));

Whether you do this by Drush or /devel/php, the result should be the same if the issue has not been resolved: 1 whatever instead of 0 whatevers.

Step 3: get a local version of the patch and apply it

In this example, we’ll look at the patch in comment #32 of our formatPlural issue, referenced above. If the issue has been resolved since this blog post has been written, follow along with another patch.

cd drupal8_core_dev_helper curl https://www.drupal.org/files/issues/2018-04-07/2273889-31-core-8.5.x-plural-index-no-test.patch -O cd ./drupal && patch -p1 < ../2273889-31-core-8.5.x-plural-index-no-test.patch

You have now patched your local version of Drupal. You can try the “0 whatevers” test again and the bug should be fixed.

Running tests

Now the real fun begins… and the “fast-track” ends.

For any patch to be considered for inclusion in Drupal core, it will need to (a) not break existing tests; and (b) provide a test which, without the patch, confirms that the problem exists.

Let’s head back to comment #32 of issue #2273889 and see if our patch is breaking anything. Clicking on “PHP 7 & MySQL 5.5 23,209 pass, 17 fail” will bring us to the test results page, which at first glance seems indecipherable. You’ll notice that our seemingly simple change to the PluralTranslatableMarkup.php file is causing a number of tests to fail: HelpEmptyPageTest, EntityTypeTest…

Let’s start by finding the test which is most likely to be directly related to our change by searching on the test results page for the string “PluralTranslatableMarkupTest” (this is name of the class we changed, with the word Test appended), which shows that it is failing:

Testing Drupal\Tests\Core\StringTranslation\PluralTranslatableMarkupTest .E

We need to figure out where that file resides, by typing:

cd /path/to/drupal8_core_dev_helper/drupal/core find . -name 'PluralTranslatableMarkupTest.php'

This tells us it is at ./tests/Drupal/Tests/Core/StringTranslation/PluralTranslatableMarkupTest.php.

Because we have a predictable Docker container, we can relatively easily run this test locally:

cd /path/to/drupal8_core_dev_helper docker-compose exec drupal /bin/bash -c 'cd core && \ ../vendor/bin/phpunit \ ./tests/Drupal/Tests/Core/StringTranslation/PluralTranslatableMarkupTest.php'

You should now see the test results for only PluralTranslatableMarkupTest:

PHPUnit 6.5.7 by Sebastian Bergmann and contributors. Testing Drupal\Tests\Core\StringTranslation\PluralTranslatableMarkupTest .E 2 / 2 (100%) Time: 16.48 seconds, Memory: 6.00MB There was 1 error: 1) Drupal\Tests\Core\StringTranslation\PluralTranslatableMarkupTest::testPluralTranslatableMarkupSerialization with data set #1 (2, 'plural 2') Error: Call to undefined method Mock_TranslationInterface_4be32af3::getStringTranslation() /var/www/html/core/lib/Drupal/Core/StringTranslation/PluralTranslatableMarkup.php:150 /var/www/html/core/lib/Drupal/Core/StringTranslation/PluralTranslatableMarkup.php:121 /var/www/html/core/tests/Drupal/Tests/Core/StringTranslation/PluralTranslatableMarkupTest.php:31 ERRORS! Tests: 2, Assertions: 1, Errors: 1.

How to fix this, indeed whether this will be fixed, is a whole nother story, a story fraught with dependency injection, mock objects, method stubs… More an adventure, really, than a story. An adventure which deserves to be told, just not right now.

The process documented process for setting up a local environment and running tests locally is, in my opinion, so complex that it can be a barrier to even determined developers.

Mediumish Blog

Drupal Themes - Fri, 04/06/2018 - 20:50

Template built specifically for Medium website design fans.