Learning journal: Enterprise Architecture

This is rather long post, and I apologise for errors in translating from latex to markdown. I will post a short lessons-learned version in the future. For context, the EA course exam in UTU was replaced with the task to write a learning journal.

Introduction

This learning journal features lessons I’ve learnt during the course TJS17 Enterprise Architecture, organized into a sections for each subject area. Material presented is based on the lectures and lecture notes of the course held in University of Turku (Heikkilä & Heimo, 2020). The journal will mostly follow the same story-line as the course, and as assigned, will consist of five main sections. Firstly, I will reflect on Enterprise Architecture (EA) in general. Then, the next lesson is about governance in EA. The third section focuses on specifically The Open Group Architecture Framework (TOGAF). Next, I discuss the course theme of the year: Agile EA. Finally, the last entry is about data architecture, one of the core topics in EA. For some background, this course is the first one I’m attending since graduating in 2018. I have worked in the industry, as a software engineer (with some software architectural duties) for a bit over five years, in a consultant/contractor SME. My clients have been SMEs, public sector organizations, and a few large enterprises. Starting 2020, I began doctoral studies in UTU focusing on privacy and data protection in software engineering. This course was a good fit for my interests: expanding my horizons from a technical focus to a more holistic view. As a note on the “methodology”, I read and listened to the lectures taking notes, and read the suggested papers. This journal is built by revisiting and reflecting on the notes and course material afterwards.

Applying Enterprise Architecture

The beginning of the course is about defining EA and presenting some aspects of it, thus I found the topic fitting for the first journal entry as well. My initial expectations for the course and my mental model of EA were somewhat aligned with how EA was presented in the first lectures, albeit on a more intuitive rather than structured level.

My experience is mainly on the software architecture perspective, so the higher level vision in the lectures is useful to me. Some business basics, such as the business model canvas I have been familiar with. In hindsight, the importance of change planning in EA took me slightly off guard. I was expecting more weight on describing businesses rather than transitions… Of course, at this moment that expectation seems silly.

Comparing enterprise and software architecture, both might seem to the outside as the discipline of describing systems. However, describing is just a means to an end: to enable change. Change is the interesting part of this, and it also contrasts to the term “architecture” in general. The person drawing a building is most often designing it, managing the process of construction, and then enjoying the fruits of their labor. For enterprises and software, the iterative approach is the way of life. While you might design a new business line or a IT service up front, it will evolve in the future (or it will die). While there are tangible parts to enterprises and IT, the potential to increase the scope is huge. Many businesses start from the garage, with little architecture save perhaps a vision in the founders mind, then grow out of nowhere. Buildings less so. This thought process is also worth considering in contrast to the EA waterfall discussed in the lectures (Pavlak, 2006). I agree that the waterfall applies to a single iteration of EA upgrading, and it ought to follow the kind of process described. The thin frameworks used to find alternatives to decide a direction and developing the chosen one further before implementing it sounds good – especially larger organizations are hard to “move fast”.

In my career so far, I have had three perspectives on EA from the inside of an organization: a small SME IT-company (C1), a government agency (C2), and a large media corporation (C3). I suppose you could also count the university, but let’s not digress. Instead, I will next apply the EA views of the course to each of these. I will try to reflect on this with regards to the lecture topics as much as I can without breaking NDAs. Regarding C1, the EA is naturally very light, and not explicit. I would guess this is common in SMEs. In the span of five years of my employment, we have evolved somewhat. Growing from ten employees to fifty does require more specificity in the organization. I would rate the EA maturity level “standardized technology stage”, but it is functional because of the small size. Essentially, there is the IT-architecture which is honed (because we are an IT-company), and the people architecture, which is loose. The people-organization was transformed from project based to a self-sufficient teams -based model. I happened to be in the working group developing the new organization. The results have been good so far: the previous structure had problems with people not feeling confident in their responsibilities and receiving competing requests.

The main EA lesson about the government organization C2 I was consulting for two years, on the other hand, was about interoperability. The largest projects of the time were two integrations between different government entities. Some of the architecture was mandated from the national level, such as the shared integration channel to pass messages through. Defining the ontologies (“koodisto” in Finnish) and the messaging protocol was a long process, and required also a future view. The entire project was an exercise in digitalization of EA, from the old paper-based bureau to a centralized digital one. This raised questions such as how the entrenched liability of office (“virkavastuu”) of independent offices could work in a centralized system, and the offices around the country had to unify their ways.

Lastly, for the previous two years of my employment, I have been working in a large Finnish media corporation, C3. It has been interesting mapping the concepts of the lectures to those in the company. In my time there, we have been through some large enterprise-level projects: divestment of multiple publications, a company wide single sign-on, merging child-companies, and a company wide publishing system change, as examples of the largest projects. The are many systems, and many companies operating their systems, and many stakeholders in the business. The beginning of this year has also been busy – the entire business line (a child company) I’ve been attached to is in the process of being sold to another media group. All in all, this enterprise has been characterized by EA transitions. The results, I would judge, have improved the maturity level, even though the changes have not been painless.

On EA governance

EA governance is a topic I chose for this journal because I don’t have much personal professional experience in it. Considering the two cycles of enterprise development: the strategic EA governance cycle, and the solution cycle, my experience is generally in the solution architecture part. I would say that I aim to move my career toward roles in the EA cycle. On a high level the process is not anything special, but the theory behind it is useful and important.

The role of the enterprise architect as the one guiding management to improve and realise their strategy, and coordinating governance is something I aspire to. Outside of the course, a good model I have found is (Hohpe, 2016) and his blog. While it is not academic content, I fully recommend it. The description of the elevator architect as moving between the penthouse of the upper management and the IT engine room. His theory is that the IT-hierarchy of traditional companies is a relic of IT being merely a cost factor, and to actually have an impact, you must pierce the levels of middle management to reach the strategy level. Addendum: in a later lecture, Hohpe was presented via a speech of his, but I’ll leave this paragraph in anyway, since it reflects my thoughts from before.

Back to the course, I would say the content assumes the elevator is “working”. It does not make sense to create governance models which have to assume challenges about power in an organization, even if they are interesting in practice. Perhaps there is some potential for future research, similar to behavioral economics… The current EA change project I am involved in, though not as the enterprise architect, involves a business line in the C3 context being sold to another group. Here we have a case about EA governance, with some unknown factors. Naturally, before the sale was finalized, the parties did due diligence and reviewed what exactly was being sold. Mapping the process to the FEAR governance model (Heikkilä, Kella, Liimatainen, & Seppänen, 2010), you could say the migration project has had all the beginning phases from objective & effectiveness assessment, resource need and availability mapping, to interoperability assessment, business modelling, and to selection of operative models. Implementation has not yet been realized, and the context differs from the FEAR approach. The iterative strategy is also clear: first to get “stable” in the new ownership, then to plan another migration to the final to-be architecture (which exists now only on a vision level).

Benefits management is not so clear in the case of the previous paragraph. I found stressing the importance of defining and monitoring the benefits of FEAR in the lectures useful. Benefits tie to the strategic level, and I find this kind of approach also useful in the solution scope on many abstraction levels. I would conjecture, that in an agile IT-development environment, on any level abstraction level, the EDM management strategy is the way to: 1) increase happiness of the managees with autonomy, 2) prevent the executives from messing with the details. This is not a critique of management or architects, but rather the conclusion from the idea that “engine room” knows their local responsibility area the best. Making use of the practical level knowledge in the strategy, being the arrow in the diagram from management level to the executives and translating the details to executive terms, is the domain of the enterprise architect.

The principles for good governance in the lectures (Dahlberg, did not find original source), I found to be a wholesome addition to the course. Each of them actually approach their topic from a humane perspective, which highlights the socio-technical aspect of EA. Combining only financial and technological values into principles would be in my opinion be less sustainable. The principle about compliance I also found suitable: data protection is a very relevant concern in information architecture for example. I won’t go much into detail about that in this journal, since that is the subject of my project work for the course. From another lecture, but somewhat on the same topic, the three schools of approaches on EA (Korhonen, Lapalme, McDavid, & Gill, 2016) range from mechanistic, to integrated, to ecosystemic philosophies. I would judge that the principles mentioned are aligned at the least with the Enterprise Integrating level of thought, overcoming the IT-only level. What kind of principles would fit into Enterprise Ecological Adaptation? The paper mentioned six principles specifically for EEA. I would say while the current ones all fit also, perhaps adding one to highlight the need for evolution might be suitable.

The Open Group Architecture Framework

Before we had the lectures on TOGAF, I had started on the course project work. This was an interesting experience. My initial plan was to map EA concepts in the literature to TOGAF terms, my reasoning being that an open architecture standard would surely be the optimal target for this. However, the entire time I studied it I felt that it was slightly off, and that the mapping might be a difficult task. I read (browsed through) (Desfray & Raymond, 2014) book about TOGAF to start with the project. Later on, the lectures about TOGAF somewhat validated my feelings. Naturally, the concepts are useful, but critique for TOGAF was also presented, which matched my thoughts. In the end, I still continued on the project work with TOGAF, convinced by the usage statistics of (White, 2018), and how the lectures mapped the other models to TOGAF terms, regardless if the TOGAF approach itself was correct.

I wish to dedicate some space to defending TOGAF. While the framework is too heavy to be practical, it has served very well as a base to form thoughts around – we can keep a higher level of abstraction, and still refer to TOGAF terms, and I would hazard that even people around the world would keep up. For example, the case for agile and iterative EA development was also presented in the TOGAF metamodel of progressive architecture.

One of the papers in the reading material critiqued the entire 4-domain, Business/Data/Applications/Technology (BDAT), conceptual model TOGAF is built on Graves, 2016). Graves’ thesis is that this view of EA is entirely from the perspective of IT-architecture, and this focus is just one out of many possible. For example, you could model the EA from a service as the base, or the organization as the base. This does make sense, and perhaps it refutes TOGAF domains as the universal way to describe EA, but it does not detract from the applicability of the ADM. Then, you can model the business in another viewpoint, but at some point you will ask: how will the IT-architecture look like? Applying a lesson from the lectures, since many businesses do view their enterprise through the BDAT lens, the path dependency means continuing to develop the EA with the same view might be correct, regardless. I agree that other perspectives are healthy and ought to be used. But if then mapping that perspective to BDAT terms results in conflicts, something is wrong in one of the models.

Another critique raises several questions regarding the viability of TOGAF in general (Kotusev, 2016). His main criticism is that no one actually practices the TOGAF ADM, and that it as a methodology does not work. Rather, TOGAF is reduced to a toolkit, which does not even include industry best practice. To use TOGAF in practice you must adapt it to the point of doing something else altogether. I am convinced by the arguments that it is not a useful practical methodology. According to the report, no one actually does TOGAF (and we dont know what doing TOGAF would actually even mean). Still, the popularity and use in theory shows, I think, that at the least it is a good baseline and a mental model for discussing and comparing architecture. Since the conclusion seems to be that everyone does their own thing, a meta-framework is useful in that way.

The most convincing critique of TOGAF in this course has been pointing out how it does not actually map to the business model very well. The scope and context of the EA is assumed to exist beforehand, before going into the ADM. Yes, the method has the step for defining the business architecture, but business activity diagrams seem much less impactful compared to business capability modeling, since the latter actually ties the business model to the technical side. Of course, nothing stops you from using any model you want in the business architecture phase, but this again ties back to the previous criticism, that no one actually does TOGAF.

Would I say I have mastered TOGAF after this course? I don’t think so. This is the downside of the heavy framework from a practical perspective. If I was now dropped into a project to enterprise architect a large system, for a company which uses TOGAF exclusively, I think I would be able to navigate existing documents/artifacts and follow the process. To lead without guidance and be confident about it, perhaps not even a dedicated TOGAF course would be enough; this kind of a beast requires getting your hands dirty for a while. Of course, claiming to master anything having only a theoretical experience is sketchy, but on a lighter framework you could at least claim to have mastered the theory. (Don’t take this paragraph as a criticism of the course, it is not.)

Agile Enterprise Architecture

The theme of the course this year, Agile Enterprise Architecture, is a natural section to include. The primary question is whether we can move from heavy planning phases to leaner, more experiment-driven enterprise architectures. The naive answer is “of course, just do the design on the fly”, but that is how you end up with balls of mud in the long term. So I’d frame the goal, not as doing less EA, rather as doing better EA. My perspective might be somewhat biased by the agile software development habits. The Agile EA part of the course featured three articles on the topic, let’s address each in turn.

To start with, lets look at the new mode of enterprise architecture management: “fast IT EAM” (Drews, Schirmer, Horlach, & Tekaat, 2017).. The idea is that enterprises in practice deploy these BizDevOps-units, cross-functional teams, which each manage their business component(s) autonomously. Still, the traditional IT -mode also persists, concurrent to the BizDevOps flow, resulting in the need for the bi-modal EA management approach. In the BizDevOps mode, the enterprise level architect works from a consulting perspective, advising the teams regarding cross-teams architecture. To me, this approach seems good, but it does have its risks. As long as the teams are split well, problems regarding data ownership, etc. might be reduced. However, if an issue such as a new business development overlaps team boundaries, it has to be handled carefully. Autonomy means that the teams have their own business objectives – if they do not align, priorities are different. This challenge is similar but on a leaner scale, to companies in a group. The importance of the same overall vision, strategy, and objectives shows again.

The Lean Enterprise Architecture Framework (LEAF) EA develop-ment fra-mework by (Hosiaisluoma, Penttinen, Mustonen, & Heikkilä, 2018) is designed as an approach for the agile environment. The principal idea of the operating model, as I understand it, is to minimize full as-is and to-be modeling and focus on incremental updates to the EA. The goal is to keep the value producing stream active. I would say that some version of this kind of operating model is in use in most SMEs, though implicitly. The development method emphasizes integrating pieces of EA from the results of iterations to the overarching architecture landscape, which is important in my opinion.

These research developments are continued in the article by (Goerzig & Bauernhansl, 2018), who present an agile EA transformation approach for SMES in a specific context. The cyclic process is similar to LEAD, but their strategy is to define the ideal architecture in a vacuum, without considering legacy systems etc. Whereas LEAD specifically builds on existing architecture. I suppose both approaches have their places, and this kind of clean slate transformation has the opportunity to make large moves in an SME. With lots of legacy systems, I’m afraid most of the ideal architecture would stay ideal, since practicality in gap analysis cloud show impossible transition paths.

What I have observed regarding this agile way of developing EA, is what I would call a “local maximum problem”. The same issue exists in software architecture also. Oftentimes, when a iterative approach establishes a solution, it becomes locked in – not hard, but in the same path dependent fashion as before. Solution $A$ in the iteration scope might be the best, but in long term there cloud be an entirely different solution $B$ to the same problem, which would make the overall architecture better. After $A$ has been established, changing $A \to B$ would be more expensive than the value you’d get. The important part (as has been stressed in the course), is to keep the overall vision in mind, especially when doing iterations. So that the task is not too simple, the converse of the issue also applies, and is hard. Focusing too hard on the long term possibilities and postponing developing specific things because a general solution is coming, is also way to lose a lot of value, in my experience. These considerations do not, of course, negate the use of agile EA development, but it highlights the need for expertise in managing the process.

The view of EA provided by (Kotusev, 2017a, 2017b) is a fresh take in my opinion. He does not show a development method per se, but the artifacts imply a progress is happening. The model is lean, since he only describes artifacts enterprises use, but there is some depth to it. Describing EA as only artifacts leaves out all of the ADM muddiness about processes which don’t happen in practice the way they are idealized. Then, how the artifacts are developed is left as implementation details. The strength is also the weakness of the model: it is only what it promises and nothing more. The axes of business and IT focus, and specific to conceptual level is a useful categorization, also.

As discussed in the lectures, one of the main tenets of DevOps is the concept of infrastructure-as-code. Scaling out micro-service server stacks, starting new environments for fast iterative versions of services, etc. are all enabled by not requiring (much) manual labor to manage IT-infrastructure. That is still squarely in the realm of technical architecture, so what would enterprise-as-code look like? It would of course encompass all the BizDevOps-things, but maybe you could also launch a new company with the push of a button. Restaurant franchises, and other such physical businesses have some of this capability. Business components with well defined interfaces help, alongside comprehensive meta-data management, so the new component can make use of enterprise resources without friction. I have yet to see in practice how this would play out in a large company. It seems that some finance or HR process always takes manual intervention, and thus time. In the future, who knows; even human labor is getting somewhat commoditized with crowd-sourcing and such.

Data Architecture

As mentioned in the lectures, data architecture is one of the most important aspects of EA, and I largely agree with this position. Eventually, it all boils down to data and meta-data; with sufficient meta-levels you can model the entire business. This does not discount the other perspectives of looking at EA, since the solely data-focused approach would essentially rebuild the other EA views from the ground-up. It does, however, hint at the power of modeling everything. Let us be practical, though.

In the same vein, the process of transforming $data \to information \to knowledge$, and again in a loop from $knowledge \to information \to data$, and so on, describes the entire data architecture development process. That is what I found interesting. Of course, that simplified process leaves a lot of blanks to fill. As noted, the complexity of data has grown in the recent years, as much that it can hardly be managed by a team, let alone a single person. Again, the myriad of roles surrounding data highlights the importance of data.

Regarding building a data architecture, I found the (Sowa & Zachman, 1992) framework useful. As it was discussed in the lectures, the static, waterfall view it provides is not the whole picture of developing EA. My take-away from the framework was not the content of the matrix, but the approach of having different levels of detail for different stakeholders, and applying each on the different domains (columns). So far my experience has been using different abstractions depending on who asks, but this explicit categorisation is helpful.

One of the materials for the topic included a presentation about information architecture, (How, 2015). The point was how to organize information, and that the best way to do it depends on the context. His thesis is that there are five different ways of organizing information: by location, alphabetically, temporally, categorically, and hierarchically. The hard part is to choose which way is the best for the present case. Highlighting the actual end goal for organizing information – good user experience – is important I feel. The converse also applies, that it is possible to get away with bad organization of information, and the system will still work. The downsides will not be apparent, but you will lose users or spend more time doing integrations. It might not be easy to sell spending a lot of time on thinking this, but defending good information architecture is important. From my experience, sometimes, when you finally crack the best way to model something, it feels like cutting the Gordian knot.

Meta-data management was, to me, the most important lesson in this topic. In my work, I am naturally experienced with defining technical meta-data for systems and integrations. This perspective of modeling explicitly business meta-data in addition to the technical side was not something I was familiar with, and found very useful. To start on the topic, we listened on a webinar about it (Burbank, 2017). The main idea I got from the webinar was the business-side metadata management. So far, I have not come across a business glossary in practice, but it sounds useful and I intend to implement one should I get the opportunity. It is normal to have a bunch of documentation, but not organized in the meta-data approach, specifically defining business terms explicitly. The idea of defining what a customer means across the enterprise, and avoiding the “I just know”-mentality that is instinctual for such concepts, is a laudable goal. However, this is not so simple in practice, as shown in (Dahlberg, Nokkala, Heikkilä, & Heikkilä, 2016). Their article demonstrates how it is actually hard to establish enterprise-wide terms as the “golden record”-type entity just does not exist: many different systems all have their valid and different concepts. Their approach is then to go down to the attribute level, and federate a shared version of them based on meta-data comparison from different data sources. It is shown that socio-contextual meta-data is (perhaps the most) important to consider when federating an attribute across systems; just technical meta-data is somewhat easier to wrangle.

Another topic from the (Burbank, 2017) webinar I enjoyed, was meta-data strategy. The idea was to progress from the perspective of business drivers, which cause challenges to stakeholders, to meta-data capabilities which help with that. These selling points are useful to articulate how establishing an actual meta-data management system gives value to the enterprise, and not just doing it for the sake of it. The main stakeholder challenges were: lack of business alignment, integrating data, data quality issues, cost of data management, no audit trails, and big data exploration. I can see how meta-data can be leveraged to improve these. Another interesting thing to consider is, that the article on big data EA (Kehrer, Jugel, & Zimmermann, 2016), collects a set of requirements for big data management which align pretty closely with the presented issues meta-data helps with. So much so, that I could almost find-and-replace “big data” to “meta data” in (Kehrer et al., 2016) and it would still be valid. Of course, big data presents unique challenges especially in data logistics, but a big data project is also a big meta-data project…

Conclusion

In conclusion, I found the course useful. The learning outcomes “apply the most common enterprise architecture design principles for developing, maintaining and changing ICT-assets and service components” and “understand the role of governance in transparent management of related activities”, I would say they were met. The course broadened by view from software to the enterprise level.

P.S. Converting the reference list below to the format of this blog was a terrible experience, do not recommend.

References

Burbank, D. (2017) Webinar: Data Modeling & Metadata Management. Dataversity. Retrieved from https://youtu.be/jyva44uHoR4

Dahlberg, T., Nokkala, T., Heikkilä J., & Heikkilä, M. (2016, 06). Data Federation with a Governance of Data Framework Artifact as the Federation Tool: Case on Clinical Breast Cancer Treatment Data.

Desfray, P., & Raymond, G. (2014). Modeling enterprise architecture with TOGAF: A practical guide using UML and BPMN. Morgan Kaufmann.

Drews, P., Schirmer, I., Horlach, B., & Tekaat, C. (2017). Bimodal enterprise architecture management: The emergence of a New EAM function for a BizDevOps-based fast IT. In 2017 IEEE 21st International Enterprise Distributed Object Computing Workshop (EDOCW) (pp. 57–64).

Graves, T. (2016). Dump the BDAT stack! Retrieved from http://weblog.tetradian.com/2016/04/11/dump-the-bdat-stack/

Heikkilä, J., Kella, T., Liimatainen, K., & Seppänen, V. (2010). FEAR Project.

Heikkilä, J., & Heimo, O. (2020). TJS17 Enterprise Architecture. University Of Turku.

Hohpe, G. (2016). 37 Things One Architect Knows about IT Transformation: A Chief Architect’s Journey.

Hosiaisluoma, E., Penttinen, K., Mustonen, J., & Heikkilä, J. (2018). Lean Enterprise Architecture Method for Value Chain Based Development in Public Sector. In ECDG 2018 18th European Conference on Digital Government (p. 86).

How, C. (2015). Yippee-IA: All You Need To Know About Information Architecture In 10 Minutes. Retrieved from https://youtu.be/TsH8y5fbfX8

Kehrer, S., Jugel, D., & Zimmermann, A. (2016). Categorizing requirementsfor enterprise architecture management in big data literature. In 2016 IEEE 20th International Enterprise Distributed Object ComputingWorkshop (EDOCW) (pp. 1–8).

Korhonen, J. J., Lapalme, J., McDavid, D., & Gill, A. Q. (2016). Adaptive Enterprise Architecture for the Future: Towards a Reconceptualization of EA. In 2016 IEEE 18th Conference on Business Informatics (CBI) (Vol. 01, p. 272-281).

Kotusev, S. (2016). The critical scrutiny of TOGAF. British Computer Society (BCS).

Kotusev, S. (2017a). Eight Essential Enterprise Architecture Artifacts. British Computer Society (BCS).

Kotusev, S. (2017b). Enterprise Architecture on a Single Page. British Computer Society (BCS)

Pavlak, A. (2006). ENTERPRISE ARCHITECTURE: Lessons from Classical Architecture

Sowa, J. F., & Zachman, J. A. (1992). Extending and formalizing the framework for information systems architecture. IBM systems journal, 31(3), 590–616.

White, K. (2018). What is TOGAF? An enterprise architecture methodology for Business