Projectional, productive Dockerfile editor

Write syntactically correct Dockerfiles every time,
with a projectional editor ensuring syntax,
providing contextual intentions and
quick fix bulbquick fixes for best practices.

Free forever, code repository

Intentions

Context-aware, intelligent suggestions

Relevant operations to the current context are always available, at an ⌥alt-↩︎enter at any time. Fix warnings and errors by choosing from multiple suggestions depending on context, or automatically apply a quickfix.

WORKDIR: not absolute path given RUN: apt-get install defaulting to recommended USER: last user for container left on root EXPOSE: port given outside range valid EXPOSE: duplicate port/protocol given RUN: apt-get missing -y requiring interactive ADD: should be COPY when not extracting tar
RUN: both wget and curl being used CMD: only last invocation affects ENTRYPOINT: only last invocation affects LABEL: merge adjacent instructions MAINTAINER: deprecated for LABEL COPY: multiple src requires trailing / FROM: unique stage names required

Refactoring

Projection-based refactoring

Instructions with references show a preview of the referenced content, fully editable. Edits in any location reflect everywhere, per character. Projections update to the most appropriate based on refactoring, such as numeric to name when a name is given.
This isn’t refactoring — this is multiple projections.

For example, expose provides VCS hints for

  • Changed/added/removed port
  • Changed/removed protocol
  • Replaced port with another

Version Control

‘Line’ changed?
See what’s changed

No more “x lines added, x lines removed”; see exactly what’s changed with context-aware gutter markers. Use the model viewer to view changes in diff views, pre-commit and from the repository history.

VCS integration for Git, Subversion and CVS

Alembicue context-aware version control commit diff
IntelliJ Docker plugin showing port bindings

Management

Integrated Docker integration

Control your local and remote Docker instances, manage images and containers, and deploy your Dockerfiles, with an intuitive visual interface.

Download v1.1.4

10.11 El Capitan or later
Developer ID signed

Requires JDK 8 on 64-bit Windows 10, 8.x, or 7 SP1 with ≥3 GB RAM
Extract .zip, run bin/alembicue.bat

Requires JDK 8 on GNOME or KDE desktop and ≥3 GB RAM
Extract tar.gz, run bin/alembicue.sh

4.3 out of 5
★★★★★
62
★★★★
17
★★★
6
★★
1
8
★★★★★
This is cool! I like the concept. Overall it is well made and I think you are very intuitive to make this app.
★★★★★
Works perfectly. Progammer is updating regulary the app.
Rotzbouf
★★★★★
Really great app and wonderful support. The one time I had a technical issue, I sent a DM to the developer and he had a fix ready in a matter of hours. Great app interface! Keep up the good work!
MissFierceness
★★★★★
I’m loving the app seriously the whole concept of it is really cool.
higheloplays
★★★★★
Absolutely love this app, and I appreciate your creation of it.
★★★★★
I’ve been using this app for a couple of weeks now and I can’t fault it so far — it does what it says on the tin! So far so good. It runs great, and looking back through the reviews, the developer actually takes the time and energy to respond to comments, good or bad.
DAH1968
★★★★★
Well… First off it actually works. Second off it’s simple and clean. There are about 2 screens you have to go through. Third. The person who manages this app has actually fixed problems that have occurred in the past. They clearly care about this app.
Phaydren

Abstract

The lack of a competent integrated development environment for Docker is a detriment to the field of containerisation and cloud computing. This project documents the development of Alembicue, a projectional and productive Docker IDE, implementing a more formally specified language definition, representative of what is necessary to create images for use with a containerisation platform. Alembicue guarantees syntactic correctness with a projectional editor mutating an abstract syntax tree, incorporating contextual intentions and quick fixes to suggest and automatically apply standards and best practices. Alembicue’s launch has been a success, being received well by the community with tens of thousands of interested visitors to the project page, and elated testimonials from those having used the application and the innovative concepts portrayed within.

Acknowledgements

I would like to thank my supervisor, Dr. Daniel Reidenbach, for his consistent support and valuable feedback on my work. I would also like to thank my family and friends for their support in my studies.

alembicatev. 1627. transitive. Chiefly figurative: to produce, refine, or transform (an idea, emotion, etc.) as if in an alembic. Cf. alembicated adj.

cuen.2 1553. 1c. A stimulus or signal to perception, articulation, or other physiological response.

Oxford English Dictionary (1989).

alembicuen. 2019. Chiefly emblematic: the juncture of the production of imagery and the container articulation process founded on the aforesaid.

With apologies to linguists.

Part 1: Foundation

This first part provides an overview to the project, beginning with its motivation, objectives and deliverables (chapter 1), followed by a review of the existing literature on this topic (chapter 2). Background information (chapter 3) is provided on the topics of containerisation, on which the language will be developed, and a comparison of language workbenches, which will be used for development of the language. The gap analysis (chapter 4) identifies issues with existing tooling with the same containerisation objectives, and discusses areas for innovation above these tools. Finally, a summary of tooling and languages chosen for use in the project provides an overview of the development environment (chapter 5).

Introduction

Overview

The aim of the project was to design, develop and release Alembicue, an integrated development environment for composing Docker images, centred around a projectional editor for Dockerfiles.

The lack of a comprehensive and integrated development environment for Docker causes issues for both developer adoption of the containerisation platform and the implementation of advanced platform functionality for professionals, as identified in gap analysis. Appropriate in-depth research of new and old editor types and functionality was necessary to establish a foundation for design with an understanding of the context of the solution.

Development was led by the choice of applicable tooling and languages including that of a language workbench forming the centre of defining and implementing a language specification. This language was derived to represent the image build language used by Docker’s image build process, while benefiting from the advantages of projectional and structural editing identified in the lessons learned from literature review. Incorporating this editor alongside orchestration mechanics to run containers based on the built images integrates an end-to-end workflow for containerisation of services, streamlining the process for novices and experts alike.

This report documents the processes involved in reaching this aim, through the completion of various objectives.

Motivation

Docker is used by more than 7 million developers worldwide (Evans Data Corp., 2018; Stack Overflow, 2019), in use by over half of all developers (Stack Overflow, 2019) and 92% of organisations (JetBrains s.r.o., 2018a). Despite this, the field is lacking a competent integrated development environment for Docker. It is left to the combined power of various tools augmenting developer knowledge to create and maintain Docker images and containers (section 3.1). These tools can be difficult to set up and use requiring a full containerisation platform installation, and may still only provide varying levels of accuracy in linting (code checking) of syntax, style and references.

A primary source of difficulty in building images for containerisation is the Dockerfile language. A Dockerfile is written by a containerisation developer and contains instructions used for the production of a Docker image, each of which creates a new build layer for the image. Docker does not provide a formal specification for the language, only officially providing a Dockerfile reference (Docker, 2018c) containing a brief description of layout, keywords and instruction parameters. To be able to write a Dockerfile, it is necessary to understand this document, which can be difficult due to the informal nature of the specification. This leads to inaccuracies and misinterpretations both in developers using the specification to write their Dockerfiles and in implementations of linting in existing tools aiming to assist developers.

This difficulty presents issues for novices and experts alike. According to the Stack Overflow Developer Survey 2019, Docker is the platform with the most interest by far from developers who are not currently developing using it (Stack Overflow, 2019). For developers looking to begin using Docker, the Dockerfile language reference can be difficult to interpret due to its informal nature. Lacking a rigid and consistent structure to the parameters and syntax used to distinguish types, it can be very easy to make a syntax error, which is only presented when the lengthy build process takes place. Furthermore, Docker is not just for development novices, with Docker ranking second in most loved platforms for development in the aforementioned survey (Stack Overflow, 2019). For experts, the advanced functionality of the Dockerfile is under-represented by tooling, yet this is the area that needs the most assistance due to its complexity. For example, multi-stage builds require careful referencing in instructions crossing stages, yet issues in tooling support and ambiguity in usage contribute to 22% of developers dreading the Docker platform (Stack Overflow, 2019).

Defining an understandable and consistent specification for the Dockerfile language, alongside an innovative editor for the language ensuring syntax compliance and providing contextual assistance in development, as part of an integrated development environment capable of supporting an entire containerisation workflow, would help novices and experts alike make the best use of the capabilities of Docker for their containerisation needs.

Benefits

The projectional editor, the core of Alembicue, edits an abstract syntax tree (AST) which directly represents the structure of the document and its content, instead of the usual text document editing, but with an interface similar to that of a standard IDE for text-based programming languages. By defining a structure of the language and behaviours for editing instances of the language, the editor enforces a requirements for properties, references and children, assisting with code completion and providing instant compliance checking for type system rules.

Since the projection in the editor is a precise and faithful reflection of the AST enforcing constraints defined in the language, it is not possible to enter invalid syntax prior to ‘parsing’ as with regular text files/editors, as there is no parsing step. For example, in a projectional editor, typing with an insertion point at a location which cannot possibly produce a valid tree with the entered characters will have no effect, as the characters are prevented from being entered by having no suitable destination.

In addition to enforcing structure and constraints based on the language, since this editor is not limited to text, the AST can be projected in alternative ways which may make understanding relationships easier. Multiple projections of nodes can be used to represent references, making parts of the file available in other locations and editable from either. Presenting the referenced node to the user ensures correct use of references, confirming that correct destinations of references are selected in the editor and by extension in the tree, and encouraging quick fixes to be performed on the secondary projection of the node, improving editing efficiency.

Containerisation is an area of active research and development, with implementations being iterated upon and new versions of the containerisation platform being released occasionally. Keywords and parameters which may have been valid for a previous version can be deprecated, and limited assistance provided in migrating to newer file versions. By clearly marking deprecated keys as such in Alembicue, the developer can see the problem immediately and fix the issue before further issues occur later on. Issues can be resolved easily by providing intentions for automatically migrating the deprecated code away from old concepts to their replacement with equivalent functionality.

Objectives

For the successful completion of this project to its aim, a number of objectives were delineated.

  1. Establish prerequisites to the project, including its motivation, methodology and work plan to ensure the project stays on track.
  2. Research and review existing literature in the field for a greater understanding of the work which has taken place prior, providing lessons learned to be built upon.
  3. Research, discuss and present conclusions, for an overview of containerisation necessary for the development of the containerisation language, and language workbenches for the implementation of the language.
  4. Establish weaknesses in existing tooling for containerisation, and areas for innovation in Alembicue, through a detailed gap analysis.
  5. Choose and set up tooling to begin development, including version control and continuous integration.
  6. Design and implement a base level of the language, with core concepts such as the abstract syntax tree, and extensibility for continued development.
  7. Design, develop and implement instructions for the language, including projectional editor components for each instruction, type system constraints and behaviours.
  8. Develop and implement an integrated development environment, incorporating the projectional editor environment and Docker orchestration to run the language.
  9. Perform suitable testing of each component of Alembicue and the overall application using established practices and methods.
  10. Release Alembicue for download and use by developers to write their own containerisation files.
  11. Evaluate the outcome of the project against each objective and the project’s overall aim, including feedback received from Alembicue’s release and possible future work to be carried out.

Deliverables

Following that which is established in project prerequisite analysis, a number of deliverables are decided upon.

  1. A simple projectional editor is the first deliverable of the project, as this is the core of Alembicue. The ability to add instructions, representing lines of code which each perform a particular task (section 7.1), is made possible with the implementation of a type system, and various supporting concepts including key- value pairs, and ports and protocols.
  2. The Alembicue language is defined and implemented, containing all instructions each with parameters, children and references applicable for the task the instruction needs to perform.
  3. Intentions, type system messages and refactoring is delivered for instructions where appropriate to make the editing experience reflect expectations of familiar text editors, as well as providing advanced functionality only possible with a projectional editor.
  4. Testing ensures functionality of Alembicue is correct, meeting the specification of the language and ensuring the usability of the software’s interface.
  5. The standalone Alembicue application is compiled for various platforms, including the Alembicue language.
  6. A project webpage is delivered to provide a location for users to download Alembicue.

Deliverable constraints

Constraints apply to the project, its objectives and deliverables, providing additional boundaries and guidance to the project’s development.

2. Scope: The language is comprised of instructions, each of which needs to be incorporated into Alembicue’s editor. These instructions are FROM, RUN, CMD, LABEL, MAINTAINER, EXPOSE, ENV, ADD, COPY, ENTRYPOINT, VOLUME, USER, WORKDIR, ARG,ONBUILD, STOPSIGNAL, and SHELL.

4. Scope: Testing requires manual testing through test plans created during development of functionality, automated unit testing for individual components and integration testing for the composition of modules.

5. Scope: Alembicue as a standalone application must support macOS, Windows and Linux to provide the greatest coverage to the developer user base.

5. Schedule: The project, including its release, must be completed prior to the deadline for the project as a whole, to be able to discuss and evaluate the release as part of the project’s report.

Methodology

Methodologies for development were decided upon early in the project which ensured all development and work product adhered to such standards.

Formal methodologies

  • Throw-away prototyping was used for the early stages of the user interface and basic functionality to quickly experiment and test different ideas and possibilities. This includes prototyping the structure of the underlying language components and of the user interface components used to display the language. While the learning that took place from the prototyping efforts was retained for main development, the work product was not.
  • Incremental building was used for the main development of the project, including the language definitions and implementation of the editor. The language was broken down into sets of functionality which could be implemented in any order, providing each prerequisite is met for each subsequent definition. From a high level, these sets were rigidly defined with a strict set of capabilities which needed to be implemented to meet the step. Such implementation was refined and components moved between stages as development progressed due to gaining a better understanding of the structure of the development and of the prerequisites needed for implementing various language and editor components.

Work plan

A work plan was developed at the start of the project and improved over the course of the design and development stages, as well as the writing of the project report. This work plan was presented in a Gantt chart (Figure 1.1) to provide an easy to understand overview of timescales for various components of the project. Annotated in the chart are the three milestones of the project: project brief, January deliverable and final deliverable. These helped define deadlines for individual comments necessary to make up those deliverables, which were included in the Gantt chart and shown by the ‘Finish to Start’ dependency relationship connection lines.