Debunking the "conventional wisdom" of data modelling, normalization, etc.

Data Modelling

2NF: The Missing Use Case

0NF, 1NF, 2NF, 4NF, Data Definition, Normalization

The previous post of “5NF: The Missing Use Case” was the first of four explaining why I gave up looking for a good data modelling book by the end of the 1980s. In this second post, I discuss what I saw as a gap in the Second Normal Form (2NF) discussion. Filling this gap helps us to appreciate how generally applicable, and consistent, Normalization is.

(More ...)

Data Modelling

5NF: The Missing Use Case

0NF, 2NF, 5NF, Candidate Key, Data Definition, ERD, Normalization

18 Aug 2023: Discussion around this post at Minimal Modeling may add to understanding of it.

When I found myself heading down the data analysis, modelling, and administration path early in my career (late 1970s), I would spend time in technical bookstores looking for texts to help me on my way. I had 4 things to check in deciding if a book was worth my while. By the end of the 1980s, I had given up on finding an useful book.

This article covers the second-highest item on my list. Subsequent posts will cover the 3rd and 4th items. The 4th, and final, post will address the most important item on my list, something which I’m astonished has not received a lot more attention. Or even any attention at all, from my experience to this day.

But I start with the poor treatment of Fifth Normal Form (5NF).

(More ...)

System Architecture

3-Tier 3/3: Implications of Changing the Application-Data Relationship

4NF, Auditability, Burroughs DMSII, Data Confidentiality, Data Integrity, Data Security, Oracle, Normalization, Programming, PostgreSQL, Schema Updates, SQL Standards, Presentation

The previous post proposed a solution to the security issue detailed in the post before it.

This post looks at a number of implications of this change, in no particular order …

(More ...)

System Architecture

3-Tier 2/3: Change the Application-Data Relationship with Stored Procedures

Data Confidentiality, Data Integrity, Data Security, Oracle, PostgreSQL, Programming

The previous post discussed the evolution of the current architecture on the right of this diagram from what previously existed on the left:

IBM 3270TerminalCICSCobolDB2CRUDClosed,ProprietaryWebBrowserApplicationCodeDataBaseCRUDOpen,ConnectedInternetFirewall
3-Tier: Closed Versus Open Technology

and asked the question of whether that has been appropriate.

I do have views on that.

But I’m only going to focus on the area in which - after the best part of 40 years doing data analysis, modelling, and administration - I feel particularly qualified to have an opinion.

(More ...)

System Architecture

3-Tier 1/3: The Expanding Attack Surface of 3-Tier Architecture

Burroughs DMSII, Data Security, Oracle, Programming

When I started out in the IT industry (although we didn’t call it that at the time), systems were abstracted at the highest level as:

UserInterfaceApplicationCodeDataBaseRequest/ResponseCreate (INSERT)Read (SELECT)UPDATEDELETE
Generic 3-Tier Depiction

(More ...)

Data Modelling

Normalize for Better Code

Normalization, Programming

My second early-career formative experience came while spending a week advising an organisation on a data strategy. To illustrate normalization, we looked at a small, self-contained project which they had in the design phase. Their data model had 5-6 tables. Over a couple of days of discussion, we restructured it into around 20 tables.

At the end they acknowledged the reasons for all the tables, while still being somewhat perplexed at the idea of so many tables for such a small project.

Some months later I was talking to someone from the organisation, and asked about that project. The person had some interesting observations.

(More ...)

Data Modelling

Normalize for Better Performance

3NF, 4NF, Normalization, Performance

The first formative experience early in my career was to address data issues in a system that was not meeting expectations. There were obvious, simple things like tuning the underlying data management software.

But more important, to me, was to normalize the data model. We did this incrementally, working with the programming team, over a couple of years.

The result was a dramatic improvement in functionality (expected) and performance (a bit of a surprise). With one round of changes a screen response dropped from a painful ~30 seconds to ~7 seconds.

I took some time to analyse why this happened. And concluded that normalization of the data was the reason.

(More ...)

Software

Wishlist: Personal WebRTC Server

WebRTC

The support of WebRTC in web browsers allows video chats to be conducted directly browser-to-browser. However, a service is still required to initiate the connection between the browsers.

But since the server has no involvement once the session is started, demands on such a server are obviously much lower than they would be if all session traffic had to transit the server.

In other words, a WebRTC connection service can be offered from considerably less powerful hardware than would be required to channel audiovisual streams for each connected chat.

(More ...)

Chronology of all Posts by Title