Resources for Getting Started with Distributed Systems | Caitie McCaffrey

Caitie McCaffrey (Microsoft); Resources for Getting Started with Distributed Systems; In Her Blog; 2017-09-07.

tl;dr → Distributed Sagas, within the .NET culture of Microsoft.

Mentions

  • Distributed SAGA
  • Simple API for Grid Applications (SAGA); In Jimi Wales’ Wiki.
  • Tao
  • Espresso
  • PNUTS
  • Transaction Processing Performance Council (TPPC, TPC)
  • Pre-materialized aggregates, a technique.

The Canon (A Canon)

Exemplars (Bloggists)

Post Mortems (After Action Reports)

Exemplars (NoSQL)

  • Bigtable, Google
  • Cassandra
  • CouchDB
  • Dynamo, Amazon
  • HBase of Apache
  • MongoDB
  • Neo4J
  • Redis
  • Riak
  • SimpleDB, Amazon

Exemplars (Full SQL)

  • MySQL
  • Oracle
  • … and so on.

Referenced

(transitively)

Argot

The Suitcase Words
  • 2-Phase Commit (2PC)
  • Available Continuous Impressive Dancing (ACID)
    Atomic, Consistent, Isolated, Durable (ACID)
  • Basically-Available, Slow Soft State, Eventually-Consistent (BASE, BASSEC)
    BASE (i.e., not ACID)
  • BLOOM, a programming language, the CALM programming language
  • Consistency As Logical Monotonicity (CALM)
  • Conflict-free Replicated Data Type (CRDT)
  • Consistency, Availability, Partition-Tolerance (CAP), (Folk-) Theorem
  • Fisher, Lynch, Patterson (FLP) Theorem
  • Liveness
  • Lots of Labor (LOL)
  • Safety
  • Serializability
  • Single System Image (SSI)
  • Read Atomic Multi-Partition (RAMP) Transactions

Previously filled.

Overview of the Digital Object Architecture (DOA) | ISOC

Overview of the Digital Object Architecture (DOA); an Information Paper, The Internet Society; 2016-10-25; 8 pages; landing.
contributor credit: Chip Sharp, scrivener

Contents

  • Introduction
  • What is the Digital Object Architecture?
  • What is a Handle?
  • Handle Resolution
  • Who Runs the Global Handle System?
  • Examples of systems based on the Digital Object Architecture/Handle System
  • Standards and the Handle System
  • Policy Considerations
  • Trademarks and Service Marks
  • Resourcs

Introduction

<quote>The Digital Object Architecture (DOA) and associated Handle System® originated at the Corporation for National Research Initiatives (CNRI) in the early 1990’s based on its work on digital libraries under contract for the Defense Advanced Research Projects Agency (DARPA).1 One of the original motivations for its design was the need to identify and retrieve information over long periods of time (on the order of tens or hundreds of years) so persistence was a critical design requirement. At the time it was developed, the Digital Object Architecture was an attempt to shift from a view of the Internet as organized around a set of hosts and the transport to reach them to a view in which the Internet was organized around the discovery and delivery of information in the form of digital objects.<quote>

Mentions

  • Digital Object Architecture (DOA)
  • associated Handle System®
    Yes, that’s a registration mark, ®.
  • Corporation for National Research Initiatives (CNRI)
  • Defense Advanced Research Projects Agency (DARPA)
  • um, like “digital libraries”
  • DONA Foundation
  • International Telecommunication Union (ITU)
  • Memoranda of Understanding (MOUs)
  • Multi-Primary Administrators (MPAs), manage the partitions of the namespace of the root servers of the top level GHR.
  • Policy Development Process (PDP)

Architecture

Digital Objects
The records, blobs of bits.
Repositories
The containers of records objects.
Handles
The names are global contextually scoped, universal, persistent.
There is a Handle Protocol, in (at least) version v2.1.
Resolution System and Registries
Like DNS, but different; maps “names” to Handles.

  • Global Handle Registry (GHR), is the root server.
  • Local Handle Services (LHS), are the regional delegates.
Handles
The names are, e.g. GUIDs, UUIDs.
  • unique
  • persistent
  • location-agnostic
  • taxonomy-agnostic

Handle

Defined as a string:
prefix “/” identifier
where
prefix is a “like a” reversed FQDN.
identifier is “like a” filename on the FQDN so referenced.
Thus handles are

  • unique
  • persistent
  • location-agnostic
  • taxonomy-agnostic
Example

For the FQDN of the U.S. Library of Congress (LOC)
FQDN: ye-auguste-national-librarye.loc.gov
Handle: gov.loc.ye-auguste-national-librarye

Registrars>

  • Coalition for Handle Services (ETIRI, CDI and CHC)
  • Communications and Information Technology Commission (CITC)
  • Corporation for National Research Initiatives (CNRI)
  • Gesellschaft für Wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG)/ePIC
  • International DOI Foundation (IDF)

Participants

DOI® System
  • International DOI Foundation (IDF)
    UK, non-profit, 1998
  • Prefix 10
  • Handle semantics is idiosyncratic, known to themselves.
  • Examples
Persistent Identifier Consortium for eResearch (ePIC)
  • [for the benefit of the] European Research Community
  • Prefix 21

Standards

RFC 3650
S. Sun, L. Lannom, B. Boesch. Handle System Overview, RFC 3650, 2003-11.
RFC 3651
S. Sun, Reilly, L. Lannom. Handle System Namespace and Service Definition, RFC 3651, 2003-11.
RFC 3652
S. Sun, S. Reilly, L. Lannom, J. Petrone. Handle System Protocol (ver 2.1) Specification, RFC 3652, 2003-11.
RFC 4452
H. Van de Sompel, T. Hammond, E. Neylon, S. Weibel. The “info” URI Scheme for Information Assets with Identifiers in Public Namespaces, RFC 4452, 2006-04.
ISO 26324:2012
International Organization for Standardization (ISO), “ISO 26324:2012 Information and documentation — Digital object identifier system“, ISO Standard 26324, 2012-06.
ANSI/NISO Z39.84-2005 (R2010)
ANSI/NISO Z39.84-2005 (R2010) Syntax for the Digital Object Identifier. (revised 2010)
X.1255
ITU-T Recommendation X.1255, Framework for discovery of identity management information, ITU-T, 2014

Trademarks and Service Marks

International DOI Foundation, Inc.
DOI, DOI.ORG, “short DOI” are registered service marks of …
DONA Foundation
DONA, GLOBAL HANDLE REGISTRY, HANDLE SYSTEM are registered service marks of…
Corporation for National Research Initiatives (CNRI)
HANDLE.NET, HDL, HDL.NET, CNRI are registered service marks of …
HDL, HDL.NET are registered trademarks of …
Internet Society
Internet Society is a registered service mark of the …

References

References

  • Documents at the DONA Foundation
  • open-stand.org
  • ANSI/NISO Z39.84-2005 (R2010) Syntax for the Digital Object Identifier. (revised 2010)
  • Corporation for National Research Initiatives, Overview of the Digital Object Architecture, July 28, 2012
  • DONA Foundation. DONA Foundation Statutes. Geneva. 2014.
  • International Organization for Standardization (ISO), “ISO 26324:2012 Information and documentation — Digital object identifier system”, ISO Standard 26324, 2012-06.
  • ITU-T Recommendation X.1255, Framework for discovery of identity management information, ITU-T, 2014.
  • R. Kahn, R. Wilensky. “A Framework for Distributed Digital Object Services”, In International Journal of Digital Libraries (2006) 6: 115.
  • Norman, Paskin. The Digital Object Identifier: From Ad Hoc to National to International. In The Critical Component: Standards in the Information Exchange Environment, edited by Todd Carpenter, ALCTS, 2015.
  • Peter J. Denning & Robert E. Kahn, The Long Quest for Universal Information Access. In Communications of the ACM, Vol. 53 No. 12, Pages 34-36.
  • RFC 3650. S. Sun, L. Lannom, B. Boesch. Handle System Overview, IETF, RFC 3650, 2003-11.
  • RFC 3651. S. Sun, S. Reilly, L. Lannom. Handle System Namespace and Service Definition, IETF, RFC 3651, 2003-11.
  • RFC 3652. S. Sun, S. Reilly, L. Lannom. J. Petrone, Handle System Protocol (ver 2.1) Specification, IETF, RFC 3652, 2003-11.
  • RFC 4452. H. Van de Sompel, T. Hammond, E. Neylon, S. Weibel, The “info” URI Scheme for Information Assets with Identifiers in Public Namespaces, IETF, RFC 4452, 2006-04.

Previously filled.

Reflections on the REST Architectural Style and “Principled Design of the Modern Web Architecture” | Fielding, Taylor, Erenkrantz, Gorlick, Whitehead, Khare, Oreizy

Roy T. Fielding, Richard N. Taylor, Justin Erenkrantz, Michael M. Gorlick, E. James Whitehead, Rohit Khare, Peyman Oreizy; Reflections on the REST Architectural Style and “Principled Design of the Modern Web Architecture; In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2017); 2017; pages 4-11 (8 pages); landing.

Performed

Reflections on REST; keynote address; performed at the 2017 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2017); by one of Roy Fielding, Richard Taylor, Rohit Khare (expect: Rohit Khare); video; 0:47:41; slides (42 slides).

Abstract

Seventeen years after its initial publication at ICSE 2000, the Representational State Transfer (REST) architectural style continues to hold significance as both a guide for understanding how the World Wide Web is designed to work and an example of how principled design, through the application of architectural styles, can impact the development and understanding of large-scale software architecture. However, REST has also become an industry buzzword: frequently abused to suit a particular argument, confused with the general notion of using HTTP, and denigrated for not being more like a programming methodology or implementation framework. In this paper, we chart the history, evolution, and shortcomings of REST, as well as several related architectural styles that it inspired, from the perspective of a chain of doctoral dissertations produced by the University of California’s Institute for Software Research at UC Irvine. These successive theses share a common theme: extending the insights of REST to new domains and, in their own way, exploring the boundary of software engineering as it applies to decentralized software architectures and architectural design. We conclude with discussion of the circumstances, environment, and organizational characteristics that gave rise to this body of work.

Mentions

  • REpresentational State Transfer (REST)
  • Computational REpresentational State Transfer (CREST)
    Computational REST (CREST)
  • Capability Uniform Resource Locator (CURL)
    Capability URL (CURL)
  • COmputAtional State Transfer (COAST)
  • Computing Resource Exchange with Security (COAST)
  • ARRESTED
  • Application Programming Interface (API)
  • Distributed Hash Table (DHT)
  • SIENA (Scalable Internet Event Notification Architectures)
  • XML
  • DHT
  • HTTP
  • REST
  • bit.ly
  • Persistsent Uniform Resource Locator (PURL)
    Persistsent URL (PURL)
  • Notifications
    • e.g. on page transitions
    • HTML ping
    • DOM, onClick, onLoad, onAnything
    • M. Thomson, E. Damaggio, B Raymor. Generic Event Delivery Using HTTP Push. RFC 8030. Internet Engineering Task Force (IETF). 2016.
  • Google Analytics
  • Google Docs
  • Google Sheets
  • AJAX
  • JavaScript
  • HTTP
    • LINK
    • UNLINK
  • Peer-to-Peer (P2P)
  • Decentralized Applications (DAPPs, dApps)
  • Client/Server
  • Web Distributed Authoring and Versioning (WebDAV, WEBDAV)
    • lock-based concurrency control
    • An RPC-based client-server centralized ile system with remote access “over HTTP”
  • Limitations of REST
    • one-shot
    • one-to-one
    • one-way
  • execution engine
  • binding environment
  • COAST
    • Capabilities
      • Services
      • Messaging
      • Interpretation
    • Claims
      • Secure remote code execution (RCE)
      • Live update
      • Novel
      • Monitoring & Traceability
      • Something about refactoring:
        Server abdication, client redelegation, server re-offering (fewer services), client reprogramming of the server.
      • Dynamic Reconfiguration
  • Group Consensus and Simultaneous Agreement (GCSA)
  • WebRTC,
  • Websockets
  • Webhooks
  • HTTP/2
  • Internet of Things (IoT)
  • Content Distrubtion Network (CDN)
  • TrueTime
  • GlobalClock
  • Apache Kafka
  • Amazon Kinesis,
  • Google Cloud Pub/Sub
  • Amazon Lambda,
  • IFTTT
  • ‘assistants’, a natural language conversational product concept, within the buzzy AI business culture. Think: Eliza, that you built in high school.
  • Cassandra
  • NoSQL
  • Federated Learning
  • Merkle Hash Trees (not MHT)
  • Bitcoin
  • <buzz>blockchain</buzz>
  • Git
    • is a decentralized in concept.
    • is not decentrlaized in practice, c.f. GitHub
  • Software-as-a-Service (SaaS)
  • Computational REpresentational State Transfer (CREST)
  • Aura
  • Nikander
  • Trickles
  • network continuations
  • Hypertext Transport Protocol (HTTP)
    • HTTP/1.1
    • HTTP/2
  • DARPA
  • NSF
  • ISR (Irvine Software Rationalization?)
  • Arcadia

Behavior, Asynchrony, State, Execution (BASE)

Concept

Adapability requires the design-time  actions…

LP1
making the parts that are subject to change identifiable, discrete and manipulable.
LP2
providing mechanisms for controlling interactions between the parts subject to change.
LP3
providing techniques for managing state.

Elaborated

  • Peyman Oreizy, Nenad Medvidovic, Richard N. Taylor. Runtime Software Adaptation: Framework, Approaches, and Styles. In Companion of 30th International Conference on Software Engineering (ICSE Companion). 2008. ACM. pages 899–910.
  • Richard N. Taylor, Nenad Medvidovic, Peyman Oreizy. Architectural Styles for Runtime Software Adaptation. In Proceedings of the Eighth Joint Working IEEE/IFIP Conference on Software Architecture and Third European Conference on Software Architecture. IEEE Computer Society, 171–180. 2009.

Exemplars

  • C2
  • CREST
  • MapReduce
  • Pipe-and-Filter
  • Event Notifications
  • “and others.”

Disambiguation

  1. within the transaction formalization of Database Theory
    • Basically Available, Soft state, Eventual consistency (BASE)
      not as used herein.
    • a consistency model wherein everything almost works
      riposte: “eventually we are all dead.”
    • Contra
      • Always Computing In Denial (ACID)
      • Atomicity Consistency Isolation Durability (ACID)
  2. within the Dynamic Software Architectures Theory, page 9.
    • Behavior
    • Asynchrony
    • State
    • Execution
  3. within the ARRESTED Theory, page 10.
    the “mindset” of a node in a distributed network.
    Best-Effort
    Others are making their best effort, as are you.
    Approximate
    There is only approximate knowledge of the state of The Other; your theory of mind is limited & foggy, slacky-latent.
    Self-centered
    Others are self-centered, as are you.
    Efficient
    Make efficient use of the only global resource: communication bandwidth to others; i.e. time is the only finite resource.

Asynchronous, Routed, REpresentational State Transfer with Estimation & Delgation (A+R+REST+E+D, ARRESTED)

  • Polling (and its inverse Asynchrony)
  • Asynchrony (and its inverse Polling)
  • Routing
  • Delegation
  • Estimation

Concept

Theory
REST+P
REST with Polling.
REST+E
REST with Estimation.
A+REST
REST with Asynchrony (callbacks).
R+REST
REST with Routing (packets).
REST+D
REST with Delegation (proxies, gateways).
ARREST
Asynchronous, Routed, REST.
ARREST+E
Asynchronous, Routed, REST, with Estimation.
ARREST+D
Asynchronous, Routed, REST, with Delgation.
ARREST+D
Asynchronous, Routed, REST, with Estimation & Delgation.
ARRESTED
A synonym for slow, yes?
Topology

The metaphor.

Poles
North
Centralized Systems
East
Estimated Systems
South
Decentralized Systems
West
Distributed Systems
Boundaries
now horizon
  • Master-Slave Styles
  • Peer-to-Peer Styles
agency boundary
  • Consensus-Based Styles
  • Consensus-Free Styles

Elaborated

Techniques

  • Bitcoin
  • and other distributed ledger schemes.

Computational REpresentational State Transfer (CREST)

Is just like functional programming.

  • The Poetry
    • mashups of Web culture are “the same as” continuations in programming language theory & culture. c.f. Scheme & SML
    • 300-series redirects are continuations

Principles

CP1
The key abstraction of computation is a resource, named by an URL.
CP2
The representation of a resource is a program, a closure, a continuation, or a binding environment plus metadata to describe the program, closure, continuation, or binding environment.
CP3
All computations are context-free.
CP4
Only a few primitive operations are always available, but additional per-resource operations are also encouraged.
CP5
The presence of intermediaries is promoted.

Concept

  • Ship code+data as a package to evaluate off-box (over there, on their box).
  • Receive code+data as a package to evaluate on-box (here on our box).
  • What could go possibly wrong here? [over there?]

Elaborations

  • Justin R. Erenkrantz. Computational REST: A New Model for Decentralized, Internet-Scale Applications. Ph.D. Dissertation. University of California, Irvine, Irvine, California, USA. 2009.
  • Justin R. Erenkrantz, Michael Gorlick, Girish Suryanarayana, Richard N. Taylor. Harmonizing Architectural Dissonance in REST-based Architectures. Technical Report UCI-ISR-06-18. Institute for Software Research, University of California, Irvine. 2006.
  • Justin R. Erenkrantz, Michael M. Gorlick, Girish Suryanarayana, Richard N. Taylor. From Representations to Computations: The Evolution of Web Architectures. In ACM SIGSOFT Symposium on The Foundations of Software Engineering (FSE). 2007. pages 255–264.
  • Roy T. Fielding. Maintaining distributed hypertext infostructures: Welcome to MOMspider’s Web. In Computer Networks and ISDN Systems, 27, 2. 1994. pages 193–204. doi:10.1016/0169-7552(94)90133-3. Series title? Selected Papers of the First World-Wide Web Conference.

Techniques

  • web mashups
  • session management
  • cookies in client/server interactions
    <quote>, and the (misplaced) role of cookies in client/server interactions</quote>
  • time-dependent resources; e.g. weather forecasts.
  • time-series responses; e.g. stock tickers.

<editorial>Why aren’t cookies necessary again? They uniquely number the consumer base. They are used to develop Measurement, Targeting, Retargeting & Profiling which are the explicit and probably only renumerative use case of the (online) media business model. Oh, right, and paywalls. And, um, public televison-type “membership drive” tip jars.</editorial>

References

There are 59 references.

Abstracted

  • Roy T. Fielding, Richard N. Taylor. Principled Design of the Modern Web Architecture. In Proceedings of the 22nd International Conference on Software Engineering (ICSE). 2000. pages 407–416. IEEE, Limerick, Ireland.

Dissertated

  • Justin R. Erenkrantz. Computational REST: A New Model for Decentralized, Internet-Scale Applications. Ph.D. Dissertation. University of California, Irvine, Irvine, California, USA. 2009.
  • Roy T. Fielding. Architectural Styles and the Design of Network-based Software Architectures. Ph.D. Dissertation. University of California, Irvine, California, USA. 2000.
  • Michael Martin Gorlick. Computational State Transfer: An Architectural Style for Decentralized Systems. Ph.D. Dissertation. Technical Report UCI-ISR-16-3. University of California, Irvine, Irvine, California, USA. 2016.
  • David Alan Halls. Applying Mobile Code to Distributed Systems. Ph.D. Dissertation. University of Cambridge, Cambridge, UK. 1997.
  • Michael Hicks. Dynamic Software Updating. Ph.D. Dissertation. Computer and Information Science, University of Pennsylvania, Philadelphia, Pennsylvania, USA. 2001.
  • Rohit Khare. Extending the REpresentational State Transfer (REST) Architectural Style for Decentralized Systems. Ph.D. Dissertation. University of California, Irvine, California, USA. 2003.
  • Mark Samuel Miller. Robust Composition: Towards a Unified Approach to Access Control and Concurrency Control. Ph.D. Dissertation. Johns Hopkins University, Baltimore, Maryland, USA. 2006.
  • Peyman Oreizy. Open architecture software: a flexible approach to decentralized software evolution. Ph.D. Dissertation. University of California, Irvine, Irvine, California, USA.
  • Emmet James Whitehead, Jr. An Analysis of the Hypertext Versioning Domain. Ph.D. Dissertation. Univ. of California, Irvine, Irvine, California, USA. 2000.

Complete

  1. T. Aura, P. Niklander. Stateless Connections. In Proceedings of the First International Conference on Information and Communication Security (Lecture Notes In Computer Science), Y. Han, T. Okamoto, S. Qing (editors), Vol. 1334. Springer-Verlag, 1997. pages 87–97.
  2. Tim Berners-Lee, Robert Cailliau, Ari Luotonen, Henrik Frystyk Nielsen, Arthur Secret. The World-Wide Web. In Communications of the ACM, 37, 8. 1994-08. pages 76–82. doi:10.1145/179606.179671.
  3. Tim Berners-Lee, Roy T. Fielding, Larry Masinter. Uniform Resource Identifier (URI): Generic Syntax. RFC 3986. Internet Engineering Task Force (IETF). 2005-01. doi:10.17487/RFC3986.
  4. Tim Berners-Lee, Roy T. Fielding, Henrik Frystyk Nielsen. Hypertext Transfer Protocol – HTTP/1.0. RFC 1945. Internet Engineering Task Force (IETF). 1996-05. doi:10.17487/RFC1945.
  5. Tim Berners-Lee, Jean-Francois Groff. The World Wide Web (a.k.a. WWW). In SIGBIO Newsletter, 12, 3. 1992-09. pages 37–40. doi:10.1145/147126.147133.
  6. Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, Karn Seth. Practical Secure Aggregation for Federated Learning on User-Held Data. In Proceedings of the NIPS Workshop on Private Multi-Party Machine Learning. 2016. landing.
  7. Antonio Carzaniga, David S. Rosenblum, Alexander L. Wolf. Design and Evaluation of a Wide-Area Event Notification Service. In ACM Transactions on Computer Systems, 19, 3. 2001-08. pages 332–383. paywall.
  8. James C. Corbett, Jeffrey Dean et. al. Spanner: Google’s Globally-distributed Database. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation (OSDI). 2012. pages 251–264. paywall, landing. slides: pptx, event: session.
  9. Chris Dixon. Crypto Tokens: A Breakthrough in Open Network Design. In His Blog, centrally hosted on Medium. 2017-06.
  10. L. Dusseault. HTTP Extensions for Web Distributed Authoring and Versioning (WEBDAV). RFC 4918. Internet Engineering Task Force (IETF). 2007.
  11. Justin R. Erenkrantz. Computational REST: A New Model for Decentralized, Internet-Scale Applications. Ph.D. Dissertation. University of California, Irvine, Irvine, California, USA. 2009.
  12. Justin R. Erenkrantz, Michael Gorlick, Girish Suryanarayana, Richard N. Taylor. Harmonizing Architectural Dissonance in REST-based Architectures. Technical Report UCI-ISR-06-18. Institute for Software Research, University of California, Irvine. 2006.
  13. Justin R. Erenkrantz, Michael M. Gorlick, Girish Suryanarayana, Richard N. Taylor. From Representations to Computations: The Evolution of Web Architectures. In ACM SIGSOFT Symposium on The Foundations of Software Engineering (FSE). 2007. pages 255–264.
  14. Roy T. Fielding. Maintaining distributed hypertext infostructures: Welcome to MOMspider’s Web. In Computer Networks and ISDN Systems, 27, 2. 1994. pages 193–204. doi:10.1016/0169-7552(94)90133-3. Series title? Selected Papers of the First World-Wide Web Conference.
  15. Roy T. Fielding. Relative Uniform Resource Locators. RFC 1808. Internet Engineering Task Force (IETF). 1995-06. doi:10.17487/RFC1808.
  16. Roy T. Fielding. Architectural Styles and the Design of Network-based Software Architectures. Ph.D. Dissertation. University of California, Irvine, California, USA. 2000.
  17. Roy T. Fielding, Gail Kaiser. The Apache HTTP Server Project. In IEEE Internet Computing. 1, 4. 1997-07. pages 88–90. doi:10.1109/4236.612229
  18. Roy T. Fielding, Henrik Frystyk Nielsen, Jeffrey Mogul, Jim Gettys, Tim Berners-Lee. Hypertext Transfer Protocol – HTTP/1.1. RFC 2068. 1997-01. doi:10.17487/RFC2068
  19. Roy T. Fielding, Julian Reschke. Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content. RFC 7231. Internet Engineering Task Force (IETF). 2014-06. doi:10.17487/RFC7231.
  20. Roy T. Fielding, Richard N. Taylor. Principled Design of the Modern Web Architecture. In Proceedings of the 22nd International Conference on Software Engineering. 2000. pages 407–416. IEEE, Limerick, Ireland.
  21. Roy T. Fielding, Richard N. Taylor. Principled Design of the Modern Web Architecture. In ACM Transactions on Internet Technology, 2, 2. 2002-05. pages 115–150.
  22. Roy T. Fielding, E. James Whitehead, Jr., Kenneth M. Anderson, Gregory A. Bolcer, Peyman Oreizy, Richard N. Taylor. Web-Based Development of Complex Information Products. In Communications of the ACM, 41, 8. 1998-08. pages 84–92.
  23. Matias Giorgio, Richard N. Taylor. Accountability Through Architecture for Decentralized Systems: A Preliminary Assessment. Technical Report UCI-ISR-15-2. Institute for Software Research, University of California, Irvine. 2015.
  24. Cristiano Giuffrida, Anton Kuijsten, Andrew S. Tanenbaum. 2013. Safe and Automatic Live Update for Operating Systems. In Proceedings of the Eighteenth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’13). ACM, New York City, New York, USA, 279–292.
  25. Y. Goland, E. Whitehead, A. Faizi, S. Carter, D. Jensen. HTTP Extensions for Distributed Authoring – WEBDAV. RFC 2518. Internet Engineering Task Force. 1999.
  26. Michael Martin Gorlick. Computational State Transfer: An Architectural Style for Decentralized Systems. Ph.D. Dissertation. Technical Report UCI-ISR-16-3. University of California, Irvine, Irvine, California, USA. 2016.
  27. Michael M. Gorlick, Kyle Strasser, Richard N. Taylor. COAST: An Architectural Style for Decentralized On-Demand Tailored Services. In Proceedings of 2012 Joint Working Conference on Software Architecture & 6th European Conference on Software Architecture (WICSA/ECSA). 2012. pages 71–80.
  28. David Alan Halls. Applying Mobile Code to Distributed Systems. Ph.D. Dissertation. University of Cambridge, Cambridge, UK. 1997.
  29. Michael Hicks. Dynamic Software Updating. Ph.D. Dissertation. Computer and Information Science, University of Pennsylvania, Philadelphia, Pennsylvania, USA. 2001.
  30. Irvine Research Unit in Software (IRUS). The Workshop on Internet-Scale Technology (TWIST). A series, 1998-2000.
  31. R. Kadia. Issues Encountered in Building a Flexible Software Development Environment: Lessons from the Arcadia Project. In Proceedings of the Fifth ACM SIGSOFT Symposium on Software Development Environments (SDE). 1992. ACM, New York, NY, USA. pages 169–180. doi:10.1145/142868.143768.
  32. Rohit Khare. Extending the REpresentational State Transfer (REST) Architectural Style for Decentralized Systems. Ph.D. Dissertation. University of California, Irvine, California, USA. 2003.
  33. Rohit Khare, Richard N. Taylor. Extending the REpresentational State Transfer Architectural Style for Decentralized Systems. In Proceedings of the 26th International Conference on Software Engineering (ICSE). 2004. IEEE Computer Society, Edinburgh, Scotland, UK. pages 428–437.
  34. Avinash Lakshman, Prashant Malik. Cassandra: A Decentralized Structured Storage System. In SIGOPS Operating Systems Review, 44, 2. 2010-04. pages 35–40.
  35. David Mazieres. The stellar consensus protocol: A federated model for internet-level consensus. Stellar Development Foundation. 2015.
  36. Mark Samuel Miller. Robust Composition: Towards a Unified Approach to Access Control and Concurrency Control. Ph.D. Dissertation. Johns Hopkins University, Baltimore, Maryland, USA. 2006.
  37. Satoshi Nakamoto. Bitcoin: A peer-to-peer electronic cash system. 2008.
  38. Peyman Oreizy. Open architecture software: a flexible approach to decentralized software evolution. Ph.D. Dissertation. University of California, Irvine, Irvine, California, USA.
  39. Peyman Oreizy, Michael M. Gorlick, Richard N. Taylor, Dennis Heimbigner, Gregory Johnson, Nenad Medvidovic, Alex Quilici, David Rosenblum. An Architecture-based Approach to Self-Adaptive Software. In IEEE Intelligent Systems, 14, 3. 1999-05 (May-June). pages 54–62.
  40. Peyman Oreizy, Nenad Medvidovic, Richard N. Taylor. Architecture-Based Runtime Software Evolution. In Proceedings of the 20th International Conference on Software Engineering (ICSE). 1998. pages 177–186.
  41. Peyman Oreizy, Nenad Medvidovic, Richard N. Taylor. Runtime Software Adaptation: Framework, Approaches, and Styles. In Companion of 30th International Conference on Software Engineering (ICSE Companion). 2008. ACM. pages 899–910.
  42. Peyman Oreizy, Richard N. Taylor. 1998. On the role of software architectures in runtime system reconfiguration. In IEE Proceedings-Software, 145, 5. 1998. pages 137–145.
  43. Dewayne E. Perry, Alexander L. Wolf. 1992. Foundations for the Study of Software Architecture. In SIGSOFT Software Engineering Notes, 17, 4. 1992-10. pages 40–52. doi:10.1145/141874.141884.
  44. Sean Rhea, Brighten Godfrey, Brad Karp, John Kubiatowicz, Sylvia Ratnasamy, Scott Shenker, Ion Stoica, Harlan Yu. OpenDHT: A Public DHT Service and Its Uses. In SIGCOMM Computing Communication Review, 35, 4. 2005-08. pages 73–84.
  45. Alan Shieh, Andrew C. Myers, Emin G. Sirer. Trickles: A Stateless Network Stack for Improved Scalability, Resilience, and Flexibility. In Proceedings of Symposium on Networked Systems Design and Implementation,/em> (NSDI), Vol. 2. USENIX Association. 2005. pages 175–188.
  46. Alan Shieh, Andrew C. Myers, Emin Gün Sirer. A Stateless Approach to Connection-Oriented Protocols. In ACM Transactions on Computer Systems, 26, 3. 2008-09. pages 8:1–8:50.
  47. James W. Stamos, David K. Gifford. Implementing Remote Evaluation. In IEEE Transactions on Software Engineering, 16, 7. 1990-07. pages 710–722.
  48. James W. Stamos, David K. Gifford. Remote Evaluation. In ACM Transactions on Programming Languages and Systems (TOPLAS), 12, 4. 1990-10. pages 537–564.
  49. Chengzheng Sun, Xiaohua Jia, Yanchun Zhang, Yun Yang, David Chen. Achieving Convergence, Causality Preservation, and Intention Preservation in Real-time Cooperative Editing Systems. In ACM Transactions on Complicating Human Interactions (HCI), 5, 1. 1998-03. pages 63–108.
  50. Richard N. Taylor, Nenad Medvidovic, et al. A Component- and Message- Based Architectural Style for GUI Software. In Transactions on Software Engineering. 1996-06. pages 390–406.
  51. Richard N. Taylor, Nenad Medvidovic, Eric M. Dashofy. Software Architecture: Foundations, Theory, and Practice. John Wiley & Sons. 2010. ASIN:B012AQ8M42: Kindle: no, paper: $151-$600.
  52. Richard N. Taylor, Nenad Medvidovic, Peyman Oreizy. Architectural Styles for Runtime Software Adaptation. In Proceedings of the Eighth Joint Working IEEE/IFIP Conference on Software Architecture and Third European Conference on Software Architecture. IEEE Computer Society, 171–180. 2009.
  53. R.D. Tennant. 1976. The Denotational Semantics of Programming Languages. In Communications of the ACM 19, 8. 1976-08. pages 437–453.
  54. M. Thomson, E. Damaggio, B Raymor. Generic Event Delivery Using HTTP Push. RFC 8030. Internet Engineering Task Force (IETF). 2016.
  55. Emmet James Whitehead, Jr. An Analysis of the Hypertext Versioning Domain. Ph.D. Dissertation. Univ. of California, Irvine, Irvine, California, USA. 2000.
  56. Emmet James Whitehead, Jr., Yaron Goland. The WebDAV Property Design. In Software, Practice and Experience 34 2004, 135–161.
  57. Wikipedia. 2017. Representational state transfer,/a>. In Wikipedia. 2017.
  58. Scott Wolchok, J Alex Halderman. Crawling BitTorrent DHTs for Fun and Profit. In Proceedings of the Fourth USENIX Workshop on Offensive Technologies (WOOT10). 2010.
  59. Gavin Wood. 2014. Ethereum: A secure decentralised generalised transaction ledger. Paper 151. Ethereum Project Yellow Papers 2014.

Previously filled.

Living on Fumes: Digital Footprints, Data Fumes, and the Limitations of Spatial Big Data | Jim Thatcher

Jim Thatcher (Clark University); Living on Fumes: Digital Footprints, Data Fumes, and the Limitations of Spatial Big Data; In International Journal of Communications (IJC); Volume 8; 2014; 19 pages; landing; previously in Proceedings of the 26th International
Cartographic Conference (ICC), 2014.

tl;dr → whereas capitalism is bad, the critical theory: sociotechnical, epistemic project, abductive processes, epistemic limits, epistemic and ontological commitments, capitalist profit motives, private corporations; frameworks of Marcuse, Pickles. You get the idea.

Abstract

Amid the continued rise of big data in both the public and private sectors, spatial information has come to play an increasingly prominent role. This article defines big data as both a sociotechnical and epistemic project with regard to spatial information. Through interviews, job shadowing, and a review of current literature, both academic researchers and private companies are shown to approach spatial big data sets in analogous ways. Digital footprints and data fumes, respectively, describe a process that inscribes certain meaning into quantified spatial information. Social and economic limitations of this data are presented. Finally, the field of geographic information science is presented as a useful guide in dealing with the “hard work of theory” necessary in the big data movement.

Mentions

  • In the introductory paragraph, cites opinements in Fast Company and Mashable as authoritative directional indicators.
  • Two problems
    1. <quote>On the one hand, rather than fully capturing life as researchers hope, end-user interactions within big data are necessarily the result of decisions made by an extremely small group of programmers working for private corporations that have [been] promulgated through the mobile application ecosystem.
    2. On the other hand, in accepting that the data gathered through mobile applications reveal meaningful information about the world, researchers are tacitly accepting a commodification and quantification of knowledge.</quote>
  • Big Data is
    • (wait for it …) very big, “large” even.
    • <quote>data whose size forces us to look beyond the tried-and-true methods
      that are prevalent at that time</quote>, Adam Jacobs.
    • Contrarianism
      • Something vague about Taylorism, Max Weber, etc.
      • Something vague about how having more data is better, or is not better.
    • The Fourth Paradigm
      1. empiricism
      2. analysis
      3. simulation.
      4. explore & exploit
    • Sources
      <quote>Most current studies describing themselves as “big data” with a spatial component revolve around two mobile software platforms [Foursquare, Twitter]</quote>

      • Foursquare
      • Twitter
      • Facebook
      • Flickr
  • Types of Data [plural of types of Datum(s)]
    • Checkin
    • Tweet
  • Livehood
  • 25% of Foursquare users link their Twitter accounts (75% don’t)
  • <quote>Finally, the reliance upon data generated with an explicit motive for profit — both for the end user and the corporation—results in epistemological commitments not dissimilar to concerns raised with regard to the knowledges and approaches privileged by GIS use. </quote>
  • <quote>This hard work of theory opens new knowledge projects within the realm of big data. For example, if the check-in is viewed as a form of disciplining technology — one that reports location to enmesh it more fully in capitalist exchange — then purposeful location fraud takes on new meaning as a potential form of resistance or protest.</quote>

Badness

  • private companies
  • profit motives
  • capitalism

Metaphors

  • Digital footprints
  • Digital fumes

Technologies

  • PostgreSQL
  • R
  • Mac (OS)

References

  • Anderson, C. (2008-06-23). The end of theory: The data deluge makes the scientific method obsolete. Wired.
  • Baker, S. (2012-01-05). Can social media sell soap? The New York Times.
  • Batty, M. (2012). Smart cities, big data. Environment and Planning B, 39, 191–193.
  • Benner, J., & Robles, C. (2012). Trending on Foursquare: Examining the location and categories of venues that trend in three cities. In Proceedings of the Workshop on GIScience in the Big Data Age 2012 (pp. 27–35). Columbus, Ohio.
  • Berry, D. M. (2011). The philosophy of software: Code and mediation in the digital age. London, UK: Palgrave Macmillan.
  • boyd, d., & Crawford, K. (2012). Critical questions for big data. Information, Communication & Society, 15(5), 662–679.
  • Brownlee, J. (2012-03-30). This creepy app isn’t just stalking women without their knowledge, it’s a wake-up call about Facebook privacy. In Cult of Mac.
  • Burgess, J., & Bruns, A. (2012). Twitter archives and the challenges of “big social data” for media and communication research. M/C Journal, 15(5).
  • Carbunar, B., & Potharaju, R. (2012). You unlocked the Mt. Everest badge on Foursquare! Countering location fraud in geosocial networks. In Proceedings of the 2012 IEEE 9th International Conference on Mobile Ad-Hoc and Sensor Systems (MASS), pages 182-190. IEEE Computer Society, Washington, DC.
  • Cerrato, P. (2012-11-01). Big data analytics: Where’s the ROI? InformationWeek: Healthcare.
  • Cheng, Z., Caverlee, J., Lee, K., & Sui, D. (2011). Exploring millions of footprints in location sharing services. In Proceedings of the Fifth International AAAI Conference on WSM. Barcelona, Spain.
  • Crampton, J. (2003). The political mapping of cyberspace. Edinburgh, Scotland: Edinburgh University Press.
  • Crampton, J. (2013). Commentary: Is security sustainable? Environment and Planning D, 31, 571–577.
  • Cranshaw, J., Schwartz, R., Hong, J., & Sadeh, N. (2012). The Livehoods Project: Utilizing social media to understand the dynamics of a city. In Proceedings of the Sixth International AAAI Conference on WSM. Dublin, Ireland.
  • Curry, M. (1997). The digital individual and the private realm. Annals of the AAG, 87, 681–699.
  • DeLyser, D., & Sui, D. (2013). Crossing the qualitative-quantitative divide II: Inventive approaches to big data, mobile methods, and rhythmanalysis. Progress in Human Geography, 37(2), 293–305.
  • Eckert, J., & Hemsley, J. (2013-04-11). Occupied Reographies, Relational or Otherwise. Presentation to the American Association of Geographers, Los Angeles, CA.
  • Exner, J., Zeile, P., & Streich, B. (2011). Urban monitoring laboratory: New benefits and potential for urban planning through the use of urban sensing, geo- and mobile-web. In Proceedings of Real CORP 2011. pages 1087–1096. Wien, Austria.
  • Farmer, C., & Pozdnoukhov, A. (2012). Building streaming GIScience from context, theory, and intelligence. In Proceedings of the Workshop on GIScience in the Big Data Age 2012. pages 5–10. Columbus, Ohio.
  • Goodchild, M. (1992). Geographical information science. In International Journal of Geographical Information Systems, 6, 31–45.
  • Goodchild, M. (2007). Citizens as sensors: The world of volunteered geography. In GeoJournal, 69(4), 211– 221.
  • Goodchild, M., & Glennon, J. A. (2010). Crowdsourcing geographic information for disaster response: A research frontier. In International Journal of Digital Earth, 3(3), 231–241.
  • Harley, J. (1989). Deconstructing the map. In Cartographical, 26, 1–20.
  • Hecht, B., Hong, L., Suh, B., & Chi, E. (2011). Tweets from Justin Bieber’s heart. In Proceedings of the ACM CHI Conference 2011. pages 237–246. Vancouver, BC.
  • Heidegger, M. (1977). The question concerning technology and other essays. W. Lovitt, Translator. New York, NY: Harper Perennial.
  • Hey, T., Tansley, S., & Toelle, K. (Eds.). (2009). The fourth paradigm: Data-intensive scientific discovery. Richmond, WA: Microsoft Research.
  • Horvath, I. (2012). Beyond advanced mechatronics: New design challenges of social-cyber systems. (Draft paper.) In Proceedings of the ACM Workshop on Mechatronic Design, Linz 2012. Linz, Austria
  • Jacobs, A. (2009). The pathologies of big data. In ACM Queue, 7(6), pages 1–12.
  • Joseph, K., Tan, C., & Carley, K. (2012). Beyond “local,” “categories” and “friends”: Clustering Foursquare users with latent “topics.” In Proceedings of ACM Ubicomp 2012. pages 919–926. Pittsburgh, PA.
  • Kingsbury, P., & Jones III, J. P. (2009). Walter Benjamin’s Dionysian adventures on Google earth. In Geoforum, 40, 502–513.
  • Kitchin, R., & Dodge, M. (2007). Rethinking maps. In Progress in Human Geography, 31, 331–344.
  • Kling, F., & Pozdnoukhov, A. (2012). When a city tells a story: Urban topic analysis. In Proceedings of ACM SIGSPATIAL 2012. pages 482–485. Redondo Beach, CA.
  • Lathia, N., Quercia, D., & Crowcroft, J. (2012). The hidden image of the city: Sensing community well-being from urban mobility. In Pervasive Computing,/em>, 7319, 91–98.
  • Laurila, J., Gatica-Perez, D., Aad, I., Blom, J., Bornet, O., Do, T., Dousse, O., Eberle, J., & Miettinen, M. (2012). The mobile big data challenge. Nokia Research.
  • Livehoods, demonstrator & promotional site. (2012).
  • Lohr, S. (2012-12-29). Sure, big data is great. But so is intuition. The New York Times.
  • Long, X., Jin, L., & Joshi, J. (2012). Exploring trajectory-driven local geographic topics in Foursquare. In Proceedings of ACM Ubicomp 2012. pages 927–934. Pittsburgh, PA.
  • Marcuse, H. (1982 [1941]). Some social implications of modern technology. In A. Arato & E. Gebhardt (Eds.), The essential Frankfurt School reader. pages 138–162. New York, NY: Continuum.
  • Martino, M., Britter, R., Outram, C., Zacharias, C., Biderman, A., & Ratti, C. (2010). Senseable city. Cambridge, MA: MIT Senseable City Lab.
  • Mayer-Schonberger, V., & Cukier, K. (2013). Big Data: A revolution that will transform how we live, work, and think. New York, NY: Houghton Mifflin Harcourt.
  • Mitchell, J. (2012-04-10). Life after death of the check-in. In ReadWrite.
  • National Science Foundation. (2012-10-03). NSF announces interagency progress on administration’s= big data initiative. Press release.
  • Noulas, A., Scellato, S., Mascolo, C., & Pontil, M. (2011). An empirical study of geographic user activity patterns in Foursquare. In Proceedings of the Fifth International AAAI Conference on WSM. pages 570–573. Barcelona, Spain.
  • Obermeyer, N. (1995). The hidden GIS technocracy. In Cartography and Geographic Information Science, 22(1), 78–83.
  • O’Sullivan, D. (2006). Geographical information science: Critical GIS. Progress in Human Geography, 30(6), 783–791.
  • Paulos, E., Honicky, R. J., & Hooker, B. (2008). Citizen science: Enabling Participatory Urbanism. In M. Foth (Ed.), Handbook of Research on Urban Informatics: The Practice and Promise of the Real-Time City. pages 414–436. Hershey, PA: Information Science Reference.
  • Pickles, J. (1993). Discourse on method and the History of Discipline: Reflections on Jerome Dobson’s 1993 “Automated geography.” In Professional Geographer, 45, 451–455.
  • Pickles, J. (1995). Ground Truth. New York, NY: Guilford Press.
  • Pickles, J. (1997). Tool or Science? GIS, Technoscience and the Theoretical Turn. In Annals of the AAG, 87, pages 363–372.
  • Presley, S. (2011). Mapping out #LondonRiots. In NFPvoice.
  • Rasmus, D. (2012-01-27). Why big data won’t make you smart, rich, or pretty. In Fast Company.
  • Sakaki, T., Okazaki, M., & Matsuo, Y. (2010). Earthquake shakes Twitter users: Real-time event detection by social sensors. In Proceedings of the 19th International Conference on World Wide Web (WWW). pages 851–860. Raleigh, NC.
  • A sea of sensors. staff; (2010-11-04). In The Economist.
  • Sheppard, E. (1993). Automated geography: What kind of geography for what kind of society? In Professional Geographer, 45, 457–460.
  • Siegel, E. (2013). Predictive analytics: The power to predict who will click, buy, lie, or die. Hoboken, NJ: John Wiley.
  • Sui, D. (2008). The wikification of GIS and its consequences: Or Angelina Jolie’s new tattoo and the future of GIS. In Computers, Environment and Urban Systems, 32, 1–5.
  • Taylor, C. (2012-11-07). Triumph of the nerds: Nate Silver wins in 50 states. In Mashable.
  • Thatcher, J. (2013-12). Avoiding the Ghetto through hope and fear: An analysis of immanent technology using ideal types. In GeoJournal. Volume 78, Issue 6, pages 967-980. paywall.
  • Thompson, C. (2012-05-10). Foursquare alters API to eliminate apps like Girls Around Me. In AboutFoursquare.
  • Twitter. (2012). Streaming API request parameters. API Documentation.
  • Weber, M. (1973 [1946]). From Max Weber (C. Mills & H. Gerth, Eds.). New York, NY: Oxford University Press.
  • Wilson, M. (2012). Location-based services, conspicuous mobility, and the location-aware future. In Geoforum, 43(6), 1266–1275.
  • Wright, D., Goodchild, M., & Proctor, D. (1997). Still hoping to turn that theoretical corner. In Annals of the AAG, 87(2), 373.
  • Xu, S., Flexner, S., & Carvalho, V. (2012). Geocoding billions of addresses: Towards a spatial record linkage system with big data. In Proceedings of the Workshop on GIScience in the Big Data Age 2012. pages 17–26. Columbus, Ohio.

Actualities

Via: backfill.

Microservices at Amazon | Munns


Chris Munns (Amazon) Microservices at Amazon; In I Love APIs 2015 Conference; 2015; 42 slides.

Mentions

  • Is it just a fad?
    tl;dr → yes, there will be a new shiny thing out in a fortnight, in a year; but that doesn’t matter.
  • Amazon, AWS
    • Apollo
    • Pipeline
    • CodePipeline
    • Code Deploy
  • Not treated
    • Latency
    • Debuggability (testability)
    • Failure models
    • again, what does one do with the spaghetti as it manifests in nested nasty HTTP cascades instead of multithreaded nested nasty call chains?

Architecture

Application assumptions

  • Infinite, independent, query flow.
  • Load Balancing
    • Global Load Balancing; i.e. DNS & CDN
    • (in colo) Load Balancing
  • API Gateway, a universal broker
  • Caching
    (in colo) Caching
  • Monitoring
    • log everything
    • dashboard everything
  • Origin Servers
  • Lambda Architecture
    Lambda Service of AWS
  • Consistency
    • Basic-Availability, Soft State, Eventual Consistency (BASE)
    • Consistency, Availability, Partition-Tolerance (CAP)

Theory

Definitional & explanatory; alphabetical.

Apologia

  • Phil Calçado; How we ended up with microservices; In His Blog; 2015-09-09.
    tl;dr

    • Ruby on Rails
    • monolith was too hard to work with
    • more features were needed
    • business success occurred (oxygen to allow a rebuild)
    • The Next SoundCloud
    • refactoring was commissioned
    • refactoring actually occurred
    • Cake!
  • Bob Rubbart (Oracle); Microservices and SOA; In Their Blog; 2015-03.
    Teaser: Similarities, differences and were we go from here.
    Bob Rhubart is community manager of Oracle Technology Network (OTN), he hosts on ArchBeat, a podcast & tweetshow.
    tl;dr → sure, Microservices is “the same” as SOA; it is SOA “done right” and suitable for <buzz>Internet of Things (IoT)</buzz>
  • The two pizza teams, In Some Blog; WHEN?
  • You build it, you run it; In Some Blog promoting Agile; WHEN?
    tl;dr → DevOps, everyone on call, no QA, launch & react.

Testimonials

Testaments to the popularity of the fad

Gilt
“From Monolith Ruby App to Distributed Scala Micro-Services” NYC Tech Talks
Nike
“Nike’s Journey to Microservices” – AWS Re:Invent 2014
SoundCloud
”Building Products at SoundCloud—Part III: Microservices in Scala and Finagle”
Capital One
“Lack Of Legacy Lets Capital One Build Nimble Infrastructure” – ThePlatform.net
Hailo
“A Journey into Microservices” – sudo.hailoapp.com

Tooling

Netflix
Some tools, at GitHub
Airbnb
SmartStack, at Their Site
Twitter
Mesos, at Apache
LinkedIn
Rest.li, at GitHub

Organization Design

  • Job Families
    the Roles & Responsibilities

    • SDE → Software Development Engineer
    • PM → Product Manager
    • TPM → Technical Program Manager
    • SE → Service Engineer
    • SDET → Software Development Engineer, Testing (a diminutive)
  • Management Theory
    same as it ever was

    Newspeak MBA-Speak
    Culture People
    Practices Processes
    Tools Technology

Microservices SOA
Substantial function requires an RPC, a network call Substantial function requires an RPC, a network call.
Many very small components Fewer more sophisticated components
Business logic lives inside a single service domain Business logic can live across multiple domains
Wire protocol must be HTTP JSON or (less preferably) XML Wire protocol is the Enterprise Service BUS (ESB), as a layer between services
(RPC) type API driven with client-deployed SDKs/Library. Middleware (?)

Actualities

Via: backfill.

Ground Truth: The Social Implications of Geographic Information Systems | John Pickles

John Pickles (editor), Ground Truth: The Social Implications of Geographic Information Systems; Guilford Press, New York; 1994-12-07; 248 pages; Guilford; Amazon: kindle $31, paper: $7+SHT.

tl;dr → (critical) Social Theory + Geographic Information Systems (GIS)

<bizarre>This book has restricted territorial rights. To order from outside the U.S. and Canada, contact Guilford for information. </bizarre>

John Pickles

  • Associate Professor in the Department of Geograph, University of Kentucky.
  • Member of the Committee on Social Theory, University of Kentucky.
Area
  • social theory
  • disciplinary history
  • regional political economy
  • geography of transition and restructuring
    • South Africa
    • Eastern Europe.

The Four Papers

  1. Sarah Elwood
  2. Nadine Schuurman, Agnieszka Leszczynski
  3. Mark Gahegan, William Pike
  4. Melissa Gilbert, Michelle Masucci

Related

Microservices | Lewis, Fowler

James Lewis, Martin Fowler; Microservices; In His Blog (Fowler’s); 2014-03-10, with revisions; ~175 paragraphs; 7200 words; with diagrams & sidebars.

Referenced

Mentioned

Promotions

Building Microservices: Designing Fine-Grained Systems | Sam Newman


Sam Newman; Building Microservices; O’Reilly Media; 2015-02-20; 290 pages; Amazon kindle: $25, paper: $25+SHT

Mentions

Concepts

  • Design Datterns
  • API Design
  • Continuous Integration
  • Continuous Delivery
  • Testing
  • Automation
  • Virtualization
  • Containers
  • Monitoring
  • Security
  • Databases (SQL, NoSQL)
  • CAP Theorem

Fairness, Isolation, Throughput (FIT): A Distributed Database Performance Tradeoff | Faleiro, Abadi

Jose M. Faleiro, Daniel J. Abadi (Yale); FIT, a distributed database performance trandeoff; Probably in Bulletin of the IEEE Computer Society Technical Committee on Data Engineering; draft; 2015-12-18; 8 pages.

tl;dr → pick any two; Fairness Isolation Throughput (FIT) is an impossible trinity; database concepts & systems are reviewed to show how they (sic) choose to avoid being FIT.  Gotta be a pun in here somewhere: FIT or FAT.

Mentions

  • Fairness, Isolation, Throughput (FIT)
  • Isolation is defined as concurrent execution in the presence of conflicts.

Exemplars

  • Calvin
  • Cassandra
  • Doppel
  • Dynamo
  • G-Store
  • Paxos
  • Riak
  • Silo
  • Spanner

Concepts

  • Atomicity
  • Availability
  • Asynchronous
  • Cache
    • Cache Coherence
  • Centralized
  • Commit
    • Group Commit
    • 2-Phase Commit (2PC)
  • Concurrency
  • Consensus
  • Consistency
  • Coordination
    • Distributed Coordination
  • Deadlock
  • Deterministic
  • Distributed
  • Fairness
  • Isolation
    • Read Atomic (RA) Isolation
    • Strong Isolation
    • Synchronization Independence
    • Weak Isolation
  • Lazy Evaluation
  • Leader
  • Liveness
  • Locality
    • Spatial Locality
  • Locking
    • 2-Phase Locking (2PL)
  • Log
  • Partitition
    • Shard
  • Partitition-Tolerance
  • Safety
  • Serializable
  • Storage
    • Stable Storage
  • Synchronous
  • Transaction
    • Distributed Transaction
    • Lazy Transaction

Referenced

In Jimi Wales’ Wiki.

References

  1. P. Bailis, A. Fekete, J. M. Hellerstein, A. Ghodsi, I. Stoica. Scalable atomic visibility with ramp transactions. In Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data (SIGMOD), pages 27–38. ACM, 2014.
  2. Basho. Riak.
  3. S. Boyd-Wickizer, M. F. Kaashoek, R. Morris, N. Zeldovich. Non-scalable locks are dangerous. In Proceedings of the Linux Symposium, pages 119–130, 2012.
  4. J. C. Corbett, J. Dean, M. Epstein, A. Fikes, C. Frost, J. J. Furman, S. Ghemawat, A. Gubarev, C. Heiser, P. Hochschild, et al. Spanner: Google’s globally distributed database. In ACM Transactions on Computer Systems (TOCS), 31(3):8, 2013.
  5. S. Das, D. Agrawal, A. El Abbadi. G-store: a scalable data store for transactional multi key access in the cloud. In Proceedings of the 1st ACM Symposium on Cloud Computing, pages 163–174. ACM, 2010.
  6. Datastax. Cql for Cassandra v1.2.
  7. G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, W. Vogels. Dynamo: Amazon’s highly available key-value store. In Proceedings of Twenty-first ACM SIGOPS Symposium on Operating Systems Principles (SOSP), pages 205–220. ACM, 2007.
  8. D. J. DeWitt, R. H. Katz, F. Olken, L. D. Shapiro, M. R. Stonebraker, D. A. Wood. Implementation techniques for main memory database systems. In Proceedings of the 1984 ACM SIGMOD International Conference on Man- agement of Data, pages 1–8. ACM, 1984.
  9. J. M. Faleiro, A. Thomson, D. J. Abadi. Lazy evaluation of transactions in database systems. In Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, pages 15–26. ACM, 2014.
  10. S. Gilbert and N. Lynch. Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services. In ACM SIGACT News, 33(2):51–59, 2002.
  11. L. Lamport. The part-time parliament. In ACM Transactions on Computer Systems (TOCS), 16(2):133–169, 1998.
  12. N. Narula, C. Cutler, E. Kohler, R. Morris. Phase reconciliation for contended in-memory transactions. In Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation (OSDI), pages 511–524. USENIX Association, 2014.
  13. A. Thomson and D. J. Abadi. The case for determinism in database systems. In Proceedings of the Conference on Very Large Data Bases (VLDB), 3(1-2):70–80, 2010.
  14. A. Thomson, T. Diamond, S.-C. Weng, K. Ren, P. Shao, D. J. Abadi. Calvin: Fast distributed transactions for partitioned database systems. In Proceedings of the 2012 ACM International Conference on Management of Data (SIGMOD), pages 1–12. ACM, 2012.
  15. A. Thomson, T. Diamond, S.-C. Weng, K. Ren, P. Shao, D. J. Abadi. Fast distributed transactions and strongly consistent replication for oltp database systems. In ACM Transactions on Database Systems (TODS), 39(2):11, 2014.
  16. S. Tu, W. Zheng, E. Kohler, B. Liskov, S. Madden. Speedy transactions in multicore in-memory databases. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, pages 18–32. ACM, 2013.

Promotions

Via: backfill.

Building Microservice Architectures | Neal Ford, ThoughtWorks

Neal Ford (ThoughtWorks); Building Microservice Architectures; In Some Venue; 2014; 80 slides.

tl;dr → Enterprise Service Bus (ESB) rides again, but with Agile, Conway, Java, JSON, HTTP, REST, CI/CD & DevOps!

Original Sources

Sam Newman; Building Microservices: Designing Fine-Grained Systems; O’Reilly Media; preview edition; WHEN?; 102 pages; free sample (final edition); 25 pages; Amazon: kindle: $31, paper: $42+SHT; O’Reilly: pdf: $43, paper: $50+SHT.

Mentions

  • Conway’s Law
    • Definition: “organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations,” as stated in the slides.
    • Melvin Conway
    • Jimi Wales Wiki.
    • Respect it: everyone must obey The Law, whether you like it or not.
      • Inverse Conway Maneuver → assemble teams around the desired architecture; the architecture will develop itself.
      • Domain Isomorphism
      • Architecture Isomorphism
  • Coupling
    • is bad
    • Efferent vs Afferent, from medicial terminology
      • Efferent → conducting away (from the center)
      • Affernet → conducting towards the center
    • Efferent Coupling is tolerable at low levels
      Fancy-speak: independent non-coordinating
  • CAP Theorem
    Choose at most two

    • Consistency
    • Availability
    • Partition Tolerance
  • Prefer BASE over ACID
    • BASE
      • Basic Availability
      • Soft-State
      • Eventual Consistency
    • ACID
      • Atomic
      • Consistent
      • Isolated
      • Durable
  • Prefer Choreography-over-Orchestration
  • Have Consumer-Driven Contracts;
  • Prefer REST-over-SOAP
    • RPC over REST is good
    • RPC over SOAP is bad
  • DevOps is good.
  • Service-Oriented Architecture (SOA)
    • is different somehow
    • is (was) J2EE, the aircraft carrier approach
    • Enterprise Service Bus (ESB), hub-and-spoke
  • Claim: Microservice is the first architectural style developed post-Continuous Delivery.
  • Something vague about “smart endpoints, dumb pipes”
  • Something vague about allowing multiple languages
    “embrace polyglot solutions where sensible”
    in practice: Java servers (Jetty), HTTP as IPC, JSON wire format, no-SQL, no-schema, Ci/CD, DevOps
  • Databases
    • Bad → Monolithic & ACID
    • Good → Lots of uncoordinated, decentralized, small, application-specific databases & hope; let BASE carry the day.
  • Something vague about decentralized governance
    Claimed <quote>Enterprise architects suffer from less pressure to make the correct choice(s) in microservice architectures.</quote>
    In practice: it’s hard to go wrong with Java/JSON/HTTP/noSQL/noschema/Ci/CD/DevOps
  • Prefer: to rewrite instead of maintain
    This has got to be somewhat controversial, but isn’t amplified in any way.
  • Scope: stay small, 10-100LOC per function; a function is now called “a service”
    This has got to be somewhat controversial, but isn’t amplified in any way [one has to marshall up a HTTP-scale network call just to get access to 10-100LOC "over there"?  Orly?  Sounds slow ... and brittle.]
  • Theory
    • no theory
      • find “a balance”
      • do “what feels right”
    • U R DOIN IT RONG → No, we’re not, it feels right & good. Go away. You’re not the boss of me.
    • Partitions
      • Domain
      • Organization
      • Transaction
      • other.
  • Taxonomy
    • Components → deployed
    • Features → released (enabled)
    • Applications → implement of routing
      • to deployed Components
      • with released Features.
    • There are no other sorts of applications.
  • Cascading Failure
    • Avoid it
    • Metaphors to slideware about
      • timeouts
      • circuit braekers
      • bulkheads
      • [firewalls?]

Nostrums

Slide 39 -80.

  • Return queries optimized for ranking & aggregation rather than for display [sure, but why?]
  • Prefer timely but partial results over slow & complete results [ahem, when appropriate]
  • Command and Query Responsibility Segregation (CQRS)
    • Query Model
    • Command Model
  • Design for Failure
  • Graceful Degradation
  • Use Monitoring
    • bad → multipane ssh terminal
    • good → Logstash+Kibana
  • Testing on Production
    • Synthetic transactions
    • Isotopes
    • Correlated (Unique) Identifiers
      a.k.a. unique order numbers
  • Service Templates, Microsoft
    • VM & (J)VM Concepts
    • A set of VMs deployed together
  • Software Configuration Management (SCM) → you will use git (you will avoid subversion)
    • lots of little repositories
    • only organized & integrated & build time → good luck!
  • Object-Relational Mapping (ORM)
    • Sooner or later you have to persist something in a real database
    • Spring
  • Testing Theory
    • Taxonomy
      • Unit Solitary
      • Unit Sociable
      • Integration
      • Boundary (Interface)
      • Component
      • Contract
      • End-to-End
    • Tooling
      • Shim
  • CI/CD & DevOps
    • Staging mustn’t diverge from Production
    • One change at a time
    • Two copies of prod: Blue & Green

Referenced

Roughly in order of appearance

Tooling

Via: backfill.

The Cloud Architect: A Necessary Evil? | DZone

The Cloud Architect: A Necessary Evil? Thomas Cozzolino; In DZone; 2015-10-24.

tl;dr → yes.  We are all cloud netizens now. On rented computers, multitenant service-bureau SaaS is the future, now!

Mentions

<buzzword level=”high”>

  • Variants
    • Application Architects
    • Data Architects
    • Enterprise Architects (EA)
    • Infrastructure Architects
    • Technical Architects
    • Solution(s) Architect
    • $X Architect for any value $X
    • etc.
  • Cloud Architect
  • Competencies
    • software design
    • integration patterns
    • security perimeters
    • Software Development Life Cycle (SLDC)
  • Frameworks (Processes)
  • Line of Business (LOB)
  • <buzzzz>Agile</buzzzz>
  • Software as a Service (SaaS)
  • Service-Oriented Architecture (SOA)
  • Bounded Context
  • Team (Organization)
    • Scrum Teams
    • Agile Teams
    • Feature Teams
    • Component Teams
  • Minimum Viable Product (MVP)
  • Patterns contra Performance
  • Portfolio
    • Application
    • Capabilities
  • Customer Resource Management (CRM)
  • Microservices
  • Representational State Transfer (REST)
  • Enterprise Resource Planning (ERP)
  • OData
    • A scheme for doing REST on (some) data
      like ODBC, JDBC, but different, more & better.
    • Since 2007.
    • OData; In Jimi Wales Wiki
    • odata.org

</buzzword>

Referenced

Via: backfill.

The Network is Reliable | ACM Queue

Peter Bailis (UC Berkeley), Kyle Kingsbury (Jepsen Networks); The Network is Reliable; In ACM. Queue; Volume 12, issue 7; 2014-07-23.

Mentions

References

Software Architecture Patterns | Mark Richards

Mark Richards; Software Architecture Patterns; O’Reilly; 2015-02; 55 pages; $0.
Teaser: Understanding Common Architecture Patterns and When to Use Them.

Mentions

  1. Layered
  2. Event-Driven
  3. Microkernel
    • plugins
  4. Microservices
    • RPC type systems: REST. SOAP, etc.
    • Message Busses
  5. Space-based (renamed from Cloud-based)
    • Something about virtualization
    • Something about middleware
    • Something about multi-tenancy
    • Something about “a grid”
    • Examples:
      • GemFire,
      • JavaSpaces,
      • GigaSpaces,
      • IBM Object Grid,
      • nCache,
      • Oracle Coherence

Analysis

Wormhole: Reliable Pub-Sub to Support Geo-replicated Internet Services | Sharma et al. (+21 others) @ Facebook, Cornell

Sharma et al. (+21 others); Wormhole: Reliable Pub-Sub to Support Geo-replicated Internet Services; In Proceedings of the 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI); 2015-05-04; 17 pages; landing, slides

Authors

Yogeshwer Sharma, Philippe Ajoux, Petchean Ang, David Callies, Abhishek Choudhary, Laurent Demailly, Thomas Fersch, Liat Atsmon Guz, Andrzej Kotulski, Sachin Kulkarni, Sanjeev Kumar, Harry Li, Jun Li, Evgeniy Makeev, Kowshik Prakasam (Facebook); Robbert van Renesse (Cornell University); Sabyasachi Roy, Pratyush Seth, Yee Jiun Song, Benjamin Wester, Kaushik Veeraraghavan, Peter Xie (Facebook)

Abstract

Wormhole is a publish-subscribe (pub-sub) system developed for use within Facebook’s geographically replicated datacenters. It is used to reliably replicate changes among several Facebook services including TAO, Graph Search and Memcache. This paper describes the design and implementation of Wormhole as well as the operational challenges of scaling the system to support the multiple data storage systems deployed at Facebook. Our production deployment of Wormhole transfers over 35 GBytes/sec in steady state (50 millions messages/sec or 5 trillion messages/day) across all deployments with bursts up to 200 GBytes/sec during failure recovery. We demonstrate that Wormhole publishes updates with low latency to subscribers that can fail or consume updates at varying rates, without compromising efficiency.

Mentions

tl;dr → like Apache Kafka, but different.

References

  • A. Adya, G. Cooper, D. Myers, M. Piatek. Thialfi: A client notification service for internet-scale applications. In Proceedings 23rd ACM Symposium on Operating Systems Principles. 2011. pages 129–142.
  • P. Hunt, M. Konar, F. P. Junqueira, B. Reed. ZooKeeper: Wait-free coordination for internet-scale systems. In Proceedings 2010 USENIX Annual Technical Conference. 2010.
  • Amazon Web Services, Inc. Amazon simple queue service. 2014.
  • Apache Software Foundation. HedWig. 2014.
  • Apache Software Foundation. Thrift. 2014.
  • R. Baldoni, C. Marchetti, A. Virgillito, R. Vitenberg. Content-based publish-subscribe over structured overlay networks. In Proceedings of the 25th IEEE International Conference on Distributed Computing Systems (ICDCS). 2005. pages 437–446.
  • R. Baldoni, A. Virgillito. Distributed Event Routing In Publish/Subscribe Communication Systems: A Survey (revised version). Technical report, Dipartimento di Informatica e Sistemistica, Universit`a di Roma la Sapienza, 2006.
  • N. Bronson, Z. Amsden, G. Cabrera, P. Chakka, P. Dimov, H. Ding, J. Ferris, A. Giardullo, S. Kulkarni, H. Li, M. Marchukov, D. Petrov, L. Puzar, Y. J. Song, V. Venkataramani. TAO: Facebook’s distributed data store for the social graph. In Proceedings 2013 USENIX Annual Technical Conference. 2013. pages 49–60.
  • A. Carzaniga, D. S. Rosenblum, A. L. Wolf. Achieving scalability and expressiveness in an internet-scale event notification service. In Proceedings 19th ACM Symposium on Principles of Distributed Computing (PODC). 2000. pages 219–227.
  • M. Castro, P. Druschel, A.-M. Kermarrec, A. I. Rowstron. Scribe: A large-scale and decentralized application-level multicast infrastructure. In Journal on Selected Areas in Communications, 20(8):1489–1499. 2002. IEEE.
  • A. Demers, J. Gehrke, M. Hong, M. Riedewald, W. White. Towards expressive publish/subscribe systems. In Advances in Database Technology-EDBT. 2006. pages 627–644.
  • P. T. Eugster, P. A. Felber, R. Guerraoui, A.-M. Kermarrec. The many faces of publish/subscribe. In Computing Surveys, 35(2):114–131. 2003. ACM.
  • Facebook, Inc. Graph search. 2014.
  • Iron.io, Inc. IronMQ. 2014.
  • J. Kreps, N. Narkhede, J. Rao. Kafka: A distributed messaging system for log processing. In Proceedings of the 6th ACM Workshop on Networking Meets Databases. 2011.
  • Y. Liu, B. Plale. Survey of Publish Subscribe Event Systems. Technical Report TR-574, Computer Science Dept., Indiana University, 2003.
  • S. P. Mahambre, S.D. Madhu Kumar, U. Bellur. A taxonomy of QoS-aware, adaptive event-dissemination middleware. In Internet Computing, 11(4):35–44. IEEE. 2007. paywall.
  • MySQL AB. MySQL.
  • R. Nishtala, H. Fugal, S. Grimm, M. Kwiatkowski, H. Lee, H. C. Li, R. McElroy, M. Paleczny, D. Peek, P. Saab, D. Stafford, T. Tung, V. Venkataramani. Scaling Memcache at Facebook. In Proceedings 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 2013. pages 385–398.
  • G. Pape. runit. 2014.
  • M. Petrovic, I. Burcea, H.-A. Jacobsen. S-ToPSS: Semantic toronto publish/subscribe system. In Proceedings 29th Conference on Very Large Data Bases (VLDB). 2003. pages 1101–1104.
  • Philotic, Inc. Beanstalk. 2014.
  • Pivotal Software, Inc. RabbitMQ, 2014.
  • M. Platania. Ordering, Timeliness and Reliability for Publish/Subscribe Systems over WAN. PhD thesis, Sapienza University of Rome, 2011.
  • V. Ramasubramanian, R. Peterson, E. G. Sirer. Corona: A high performance publish-subscribe system for the world wide web. In Proceedings 3th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 2006. pages 15–28.
  • W. Rao, L. Chen, A.-C. Fu, H. Chen, F. Zou. On efficient content matching in distributed pub/sub systems. In Proceedings of IEEE INFOCOM. 2009. pages 756–764.

Via: backfill

Software Architecture Conference (SoftwareArchitectureCon)

Software Architecture Conference, O’Reilly Media, Boston, 2015-03-16.

Mentions

  • Microservices
  • Cloud Foundry
  • The MEAN Stack
    • MongoDB
    • ExpressJS
    • AngularJS
    • NodeJS
  • Model-View Controller (MVC)
    • Server-side MVC
    • Client-side MVC
  • Heroku, a Salesforce.com company
  • Conway’s Law
  • DevOps
  • Semantic Triples
  • Continuous Delivery
  • Continuous Deployment
  • Event Sourcing (vs REST vs CRUD)
  • Reactive Systems
  • Containers
  • Apache Mesos

Actualities

From the Schedule

Via: backfill

The Art of Systems Architecture | Eberhardt Rechtin, Mark Maier

The Art of Systems Architecting, Third Edition
Mark W. Maier; The Art of Systems Architecting, Third Edition; CRC Press; 2009-01-06; 472 pages; kindle: $98, paper: $103+SHT; ACM.

The Art of Systems Architecting, Second Edition
Eberhardt Rechtin, Mark W. Maier; The Art of Systems Architecting, Second Edition; 2000-06-28; 344 pages; kindle: $18, paper: $0.01+SHT; First Edition; 1997; 308 pages.

Systems Architecting of Organizations: Why Eagles Can't Swim

Eberhardt Rechtin; Systems Architecting of Organizations: Why Eagles Can’t Swim; CRC Press; 1999-07-27; 280 pages; kindle: no, paper: $47+SHT; ACM.

Eberhard Rechtin; “The Extension of Systems Architecting to the Architecting of Organizations”; In Proceedings of the 1999 International Conference on Software Engineering; 1999-05-22; page 622; paywalled.

Reviews

Sebastian Fixson; Book Review of The Art of Systems Architecting, Mark Maier, Eberhardt Rechtin; Course ESD.83, MIT, David A. Mindell; 2000-11; 8 pages. tl;dr => 95% overview of the table of contents, didn’t approve of the separation of Chapter 12 (Political Concerns) as a separable axis.

Fleming Ray; Book review of Systems Architecting of Organizations: Why Eagles Can’t Swim, Eberhard Rechtin; Course ESD.83, MIT; David A. Mindell; 2011-11; 3 pages. tl;dr => basic overview.

Table of Contents

from the first edition

  • Part One Introduction — The Art of Architecting
    • Chapter 1 Extending the Architecting Paradigm
    • Chapter 2 Heuristics as Tools
  • Part Two: New domains, New Insights
    • Chapter 3 Builder-Architected Systems
    • Chapter 4 Manufacturing Systems
    • Chapter 5 Social Systems
    • Chapter 6 Software and Information Technology Systems
    • Chapter 7 Collaborative Systems
  • Part Three Models and Modeling
    • Chapter 8 Representation Models and System
    • Chapter 9 Design Progression in System Architecting
    • Chapter 10 Integrated Modeling Methodologies
    • Chapter 11 Architecture Frameworks
  • Part Four: The Systems Architecting Profession
    • Chapter 12 The Political Process and Systems
    • Chapter 13 The Professionalization of Systems
  • Appendices
    • Appendix A: Heuristics for Systems-Level Architecting
    • Appendix B: Reference Texts for Suggested for
    • Glossary

Concepts

  • Builder-Architected Systems
  • The Form-First Paradigm contra the Function-First Paradigm
  • Open Source [Chapter 7]
  • Uncertainty of end purpose.
  • Unstated assumptions.
  • Successful architectures are propietary, but open [page 65].

Highlighed

Brenda Forman (USC, Lockheed Martin); The Political Process and Systems Architecting; Chapter 12; pages 250-261.

Aphorisms

  • <quote>If the politics don’t fly, the system never will.</quote>
  • <quote><snip/>the political process bases its decisions on very different information than the engineering process does. Learn to satisfy both those sets of requirements by plan.</quote>

Facts of Life

  1. Politics, not technology, sets the limits of what technology is allowed to achieve.
  2. Cost rules. Yet affordability is decided by whichever side has the most votes.
  3. A strong, coherent constituency is essential.
  4. Technical problems become political problems. There are no purely technical problems.
  5. The best engineering solutions are not necessarily the best political solutions.

Approaches

  • With few exceptions, schedule delays are accepted grudgingly; cost overruns are not, and for good reason.
  • The Triage: when there is only so much that can be done, let the dying die. Ignore those who will recover on their own. And treat only those who would die without help.
  • The most important single element of success is to listen closely to what the customer (in this case, the Congress) perceives as his requirements and to have the will and ability to be responsive (J.E. Steiner, The Boeing Company, 1978)
  • Look out for hidden agendas.
  • And so on.

Skills

  • Understand the client.
  • Be informed.
  • Understand different viewpoints

Referenced

  • Social Systems, Facts vs. Perceptions; a book; cited page 258.
  • Jonathan Losk, Tom Pieronek, Kenneth Cureton, Norman P. Geis; Some Paper; WHEN?
    Based on the Myers-Briggs Type Indicator (MBTI), the paper strongly suggestsasserts that the preferred personality type for architecting team membership is NT.

Via: backfill

Distributed Systems and the End of the API | Chas Emerick

Chas Emerick; Distributed Systems and the End of the API; In Some Blog; 2014-04-23 (or thereafter).

Other Materials
Commentariat

Table of Contents

Mentions

Referenced

Actualities

500px-NetworkTopologies.svg