Architecting for Continuous Deployment and DevOps

Project Context: Continuous deployment (CD) and DevOps has emerged as popular software development paradigm, which tries to establish a strong connection between development and operations teams. Adopting and supporting CD/DevOps for industrial organizations involves a large number of challenges because organizational processes, practices, and tool support may not be ready to support the highly complex and challenging nature of DevOps. It is argued that one of the most pressing challenges which the organizations may encounter is how software applications should be architected to support CD/DevOps practices such as Continuous Delivery, Continuous Testing, Continuous Monitoring and Optimization and Continuous Deployment. Due to the highly complex and challenging nature of DevOps practices, developing such kind of software applications entail high complexity during the development lifecycle.

The main objective of this project is to develop and evaluate a new generation of framework, reference architectures, guidelines, and tools to support architectural decision making process in the context of DevOps. The framework and associated tool will be built upon the large-scale empirical study involving practitioners from industrial collaborators. This framework and tools will help document several aspects of DevOps-specific decisions, patterns and reusable components. Since most of DevOps practices (e.g., deployability, testability and security) can be considered as quality attributes, the framework should assist software architect to make the right and informed architectural decisions to meet DevOps practices and make a trade-off between them as well as guide the architecting for DevOps. To evaluate the effectiveness and usability of this framework and tooling support, we will conduct several case studies with both industrial software systems and big data analytics applications.

Architecture and Knowledge Support for Big Data Systems

Project Context: Big Data Systems (BDS) (i.e., data-intensive applications) have become one of the key priority areas for all sorts of organizations (i.e., private or public)[1],[2]. Nowadays public or private organizations are expected to leverage proprietary and open source data for different purposes such as business strategies, social networking, securing citizens and societies, and promoting scientific endeavors. To effectively and efficiently capture, curate, analyze, visualize and leverage such a large amount of data, a significant number of efforts are being invested to invent new and innovative techniques and technologies for supporting several functions of Big Data systems such as data capture and storage, data transmission, data curation, data analysis, and data visualization. One of the key challenges of designing, deploying, and evolving Big Data systems is designing and evaluating appropriate architectures that can support continuous development and deployment of Big-data systems. Hence, there is a vital need of developing and rolling out approaches and technologies for identifying, capturing critical knowledge and expertise, and making it available for transfer and reuse across various Big-Data systems projects.

We plan to build and evaluate a knowledge base to support the systematic design and evaluation of BDS. For this project, this knowledge base means reusable design knowledge and design artefacts and a tooling infrastructure for managing and sharing the knowledge and artefacts. The design knowledge will consist of a set of design principles, meta-models of describing BD systems’ core functional and non-functional properties, design patterns, other reusable design artefacts and intelligent algorithms to explore the available design artefacts.

[1] DRAFT NIST Big Data Interoperability Framework: Volumen 6 Reference Architecture, Draft Version 1, April, 2014.

[2] Data-intensive applications, challenges, techniques and technologies: A Survey on Big Data by C. L. P. Chen and C. Zhang, Information Sciences, 275, pp. 314-347, 2014.

Automated detection and prevention of data exfiltration

Project Context: Increasing volume and value of data and modern day work arrangements where workers are mobile provide motivation and weak links for cyber attacks. Researchers and practitioners are becoming convinced that one of the best strategies should be based on the assumption that there will be a weakest link to be exploited for cyber security attacks. We assert that appropriate architectural designs can play a critical role in supporting automated mechanisms to detect and disrupt data leakage attacks. This project will focus on identify and classifying data exfiltration challenges that can be addressed at the architecture level and devising appropriate architectural strategies by applying design patterns and tactics. The solutions will be demonstrated by building appropriate prototypes. The devised solutions are expected to adapt to different types of data exfiltration attacks and introduce appropriate mechanisms for detecting, mitigating and preventing data exfiltration attempts. The devised solutions should also be able to support some sort of recovering in case of a data exfiltration attacks.

Middleware for Managing Data location, Security, and Privacy

Project Context: One of the key barriers to widespread adoption of cloud computing is lack of fine-grained control mechanisms on the location, security, and privacy of data individuals and organizations can store, process, or move using cloud technologies. The users also need to know and control how cloud service providers enable them to fulfill different legal, organizational, and social compliance obligations. Our research will aim to develop an integrated framework to provide theoretical founding and practical strategies for designing and implementing a middleware for providing fine-grained management of data location, security, and privacy. To achieve our goal of providing a policy driven middleware, this work will combine research on data location requirements, domain-specific languages for specifications of security and privacy constraints, and principles for designing policy driven adaptive middleware.

Collaborative Workspaces for Crowd-based Design and Validation of Industry Systems

Project Context: The emergence of crowdsourcing phenomenon has opened up many venues for soliciting and providing knowledge-intensive services. In the context of design and validating industry software systems, an organization’s internal and external crowd can provide immense amount of knowledge on a very short notice. Whilst the phenomenon is gaining increasing popularity, the underpinning theoretical foundations, business models, and supportive technological infrastructure are in their infancy stages. Our work will aim at developing cloud-enabled infrastructure for supporting experimentation for developing theoretical concepts, providing virtualized multi-tenant collaborative workspaces for design and validation of industrial systems while maintaining the required level of security and privacy for unknown workforce. Our research will also focused on understanding the challenges involved in ensuring quality of the work done by the members of a crowd and devising and evaluating appropriate strategies for achieving the required quality level by gaining appropriate alignment of socio-technical congruence (i.e., alignment between social and technical factors) that is considered to have positive impact on the quality of software development tasks.

Requirement Engineering for Security

Security is one of the biggest challenges facing organisations in the modern hyper-connected world. A number of theoretical security models are available that provide best practice security guidelines and are widely utilised as a basis to identify and operationalise security requirements. Such models often capture high-level security concepts (e.g., whitelisting, secure configurations, wireless access control, data recovery, etc.), strategies for operationalising such concepts through specific security controls, and relationships between the various concepts and controls. The threat landscape, however, evolves leading to new tacit knowledge that is embedded in or across a variety of security incidents. These unknown knowns alter, or at least demand reconsideration of the theoretical security models underpinning security requirements. In this paper, we present an approach to discover such un-known knowns through multi-incident analysis. The approach is based on a novel combination of grounded theory and incident fault trees. We demonstrate the effectiveness of the approach through its application to identify revisions to a theoretical security model widely used in industry.

Designing and Implementing a (Secure) Continuous Deployment Pipeline

Since there is no a standard continuous delivery and deployment pipeline in the industry, the organizations always have challenges to design and implement their continuous deployment pipeline. Therefore, a large number of continuous deployment pipeline models can be proposed and definitely there exists an extensive trade-offs to select which pipeline model is more suitable to a given organization based on many criteria such as organizational context, culture, security concern, etc. Furthermore, a wide range of open-source and commercial tools can be employed in the different stages of a pipeline. It is clear that those tools are not easily integrated together and the organizations have to come up with how to integrate the existing tools in order to make a tool chain to be used in the pipeline. So, this project is aimed at designing and implementing a continuous deployment pipeline. The students are expected to do the following tasks:

  1. Design the architecture of continuous deployment pipeline: The students should define the structure of pipeline, what the different stages of the pipeline are and what the functions of each stage are.
  2. Selecting appropriate tools to be used in the pipeline: A tool chain is needed to support continuous deployment practices. The students need to understand the existing continuous deployment tools (e.g., Jenkins, Puppet, Chef, Maven, automation test tools, etc.).
  3. Implementing and testing the pipeline through a given application: In the last step, it is needed to evaluate the designed pipeline using a toy example.

Migrating monolithic applications to microservices architecture style

Continuous Delivery/Deployment practices. At the best state, the goal is to break down (monolithic) applications into independently deployable and small services to support and enable continuous deployment and reduce dependencies between teams. To the best of our knowledge, the organizations do not employ any specific criteria or frameworks to achieve this goal. According to literature, the following potential criteria are important to define and determine small and independent deployable components/services:

  • A component/service is small if it can be scaled independently.
  • A component/service is small if it can be deployed independently.
  • A component/service is small if it can be tested independently.
  • A component/service is small if it can be modified independently

However, a number of practitioners [1] discussed how monolithic applications can be migrated to continuous delivery/deployment practice through microservices architecture, but to the best of my knowledge, there is only one academic paper in this regards [2]. In this project, the students are asked to decompose a monolithic application into small and independent services based on the above-mentioned criteria and other practices and principles that have been extracted from [3].

[1] Adopting Microservices at Netflix: Lessons for Architectural Design, Available at https://www.nginx.com/blog/microservices-at-netflix-architectural-best-practices/
[2] A Balalaie, A Heydarnoori, P Jamshidi, Microservices Architecture Enables DevOps: an Experience Report on Migration to a Cloud-Native Architecture, IEEE Software 2016.
[3] Mojtaba Shahin, Muhammad Ali Babar and Liming Zhu, The Intersection of Continuous Deployment and Architecting Process: Practitioners’ Perspectives, 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), 2016.

A tool to leveraging, customizing and visualizing data generated in continuous deployment pipeline for different stakeholders (e.g., architect)

A continuous deployment (CD) pipeline is composed of a number of open sources and/or commercial tools. During each steps (e.g., build, package, test etc.) of CD pipeline, a lot of information is produced. Although each tool may provide dashboard functionality to visualize the information (e.g., number of failure), but such functionality has two problems: 1) the information is NOT customized for stakeholders (e.g., QA team) based on their needs; and 2) each dashboard functionality can be considered as separated data islands, in which cannot give the whole picture of application in CD pipeline. [1, 2] fairly investigated this problem; however, the main themes of these papers are about leveraging the produced data for developers and testers in continuous integration system rather than CD pipeline. In this project students are expected to develop a tool to leverage and aggregate data produced in CD pipeline to help architect make informed data-driven decision.

[1] SQA-Mashup: A Mashup Framework for Continuous Integration, Information and Software Technology, 2015.
[2] SQA-Profiles: Rule-based Activity Profiles for Continuous Integration Environments, SANER 2015.

How much Open Source Communities concern about deployability and other DevOps-related quality attributes?

There are two papers that investigated deployability as an emerging quality attribute in DevOps context. In [1], authors examined design decisions made to satisfy deployability from three projects and then designed deployability tactics tree. In [2], the authors investigated the scenarios related to different quality attributes in 31 projects during 15 years of Architecture Trade-off Analysis Method data. They only reported how much quality attributes (e.g., testability and deployability) have been covered in the studied projects.

Through this project, we are going to investigate requirements, architectural information, and commit messages from mailing list, issue/bug tracking systems and online discussion in open source projects to understand how open source communities consider and improve deployability of an application and other DevOps-related quality attributes.

[1] Bellomo, S., Ernst, N., Nord, R. Kazman, R.; Toward Design Decisions to Enable Deployability: Empirical Study of Three Projects Reaching for the Continuous Delivery Holy Grail, Dependable Systems and Networks (DSN), 2014 44th Annual IEEE/IFIP International Conference on
[2] Bellomo, S., Gorton, I., Kazman, R.; Toward Agile Architecture Insights from 15 Years of ATAM Data, IEEE Software, 2015.
[3] Mehdi Mirakhorli, Jane Cleland-Huangy, Detecting, Tracing, and Monitoring Architectural Tactics in Code, IEEE Transaction on Software Engineering, 2016.
[4] Mining Energy-Aware Commits, MSR 2015.
[5] How Android App Developers Manage Power Consumption? An Empirical Study by Mining Power Management Commits, MSR 2016.