Layouts

The traditional way of using templates is: First, you jump into your code, initialize vars, and then, execute a template that includes some sort of pseudo-language to insert those vars inside markup.
Let's start with HTML.HTML is a language that, by reading its own and related (css,etc) specs, it's composed by tags that are meant to stylize what's inside them.Only a bunch of those tags have some sort of notion of what it's outside them, so they are part of a bigger structure: &lt:td>, inputs in a <form> , <li> etc.
But websites are not made of a bunch of independents, stacked <div>'s .
There are menus, sidebars, headers, footers, and quite a few different ui elements, each one of those composed by a few divs, and each one of those divs has a semantic role , in terms of that ui element.
Divs are to UI elements what assembler is to C.We're creating user interfaces using the most simple elements possible.There are no intermediate abstractions.We (struggle to) create a menu for our website, by combining divs and css.But that menu has only existence once it's rendered, and the user interprets those coloured divs or uls, as a "menu".
Nowhere in our code (except comments), has assigned that role to that bunch of markup.Once you start creating the user interface, you directly jump in the HTML and CSS madness.

So, it's not strange that Javascript has taken the role of creating the user interface.Libraries / frameworks like The Dojo Toolkit do a great work leveraging the concepts a programmer has to use to create user interfaces.
I think that's a big slap in the face to HTML.Sorry, HTML5, you're so great at doing things Flash was doing a decade ago, and still, fail miserably at giving any better support for creating serious UIs.

So, HTML has basically been reduced to the <div> tag, that you can convert to anything, using CSS (then, some folks still try to give "semantic" meaning to HTML).And, on top of that, use javascript to do the real rendering.
To see things in perspective: it's like we were writing components in the old Visual Studio 4, instantiating objects, but also, editing the .res files by hand.We're creating user intefaces programatically (javascript), and fighting with divs, css and browser incompatibilities.

So, after all that, is there a way to describe an user interface, in a more abstract way, and without resorting to javascript-based UI construction (so we keep javascript's small resources for our own purposes instead sustituying the browser rendering engine?)

What about this:

   [_MENU]
        [_MENUGROUP]
                [_TITLE]Main Navigation[/TITLE]
                [_OPTIONS]
                          [_OPTION][_LABEL]Home[/LABEL][_URL]http://...[/URL][/OPTION]
                          [_OPTION][_LABEL]...[/LABEL][_URL]http://...[/URL][/OPTION]
                          ...
                 [/OPTIONS]
        [/MENUGROUP]
        [_MENUGROUP]
                 [_TITLE]Support[/TITLE]
                 ....
   [/MENU]                                                  

Again, just by reading the above code, you'd know what's supposed to do.If i had written all that in its <div>, <ul>, <a>, etc equivalents, with all their css classes, attributes and so, you'd still be looking at the code trying to figure out what's its purpose. And, again, i've said nothing about how this code should be rendered.Is it a navigation bar?Is it some sort of tabbed widget? That doesnt matter; again, we're using abstractions we all know, like "Menu", and leaving out implementation details.That will be solved at a lower level. I've attempted to write template engines (like this one) to support this kind of structures, and, although they're not perfect, they did the job.But now, i think i've found the right way to do it...And hope to post some code soon! Stay connected!

Resurrecting Data Types.

The basic data types found in most languages are almost always the same.You'll find ints,strings,floats,doubles and not many more.Well, saying that something is an "int" isnt exactly a great jump in terms of describing the nature of the data you want to store in it.
That's why i like how it's usual to find in C code the use of typedef to create aliases of basic types.So you can define "Height" as an alias of "int", that helps a lot in terms of code readability.
If you need is to add more "personality" to your data, the next step are classes (i know, C/C++ also have structs).

A typical piece of data in the web, is an user login name.Looking at the options a programmer has in his hands (basic data types), he will , in almost all cases, use a string as its type.Not many will use a class to model it, specially when in the User table, the Login field is a varchar.
Also, ORM's that use the table's schema to gather information about a class, will also conclude that Login is a string.

But "Login" isn't just a string.It has many properties.For starters, it has a maximum length, and also, a minimum length.While the first property can be gathered by an ORM from the "User" table schema,what about the second?. A valid "Login" also has to match a certain regular expression.And, from the User table perspective, a "Login" field must be unique, and cannot be NULL.
Not only that.A "Login" field, when shown in a form, will be rendered using an input of type "text".And, possibly, when shown anywhere else, will have a typical action, as visiting the user profile, or something else.

But, the most we usually say about the "Login" field is : varchar(15).

Of course, in a web site, the "Login" field will be validated somewhere, shown somewhere, and stored in the user table in some other place.
But, all those actions, will not be executed as inherent abilities of the Login data type, (because it doesnt exist), but as a certain form validation code, or as a certain template rendering process.
There is no guarantee that, given two forms that use the Login field, both of them will restrict the user input to 15 characters.It's in the programmer hands the responsability of adding that restriction.If later it's decided the Login fields should allow 20 characters,it's again in the programmer hands to change those form inputs to allow the new size.

Everytime i see a framework claiming to be DRY, i check for this feature: can i specify that my "login" field has a minimum of 4 characters and a maximum of 15,and uses this regular expression, so it'll take care of drawing the right input, tell the user about those limits, and do all the syntax check before letting my code start using it?And will the Login field be represented in the DB as a varchar(15) too?

If that framework cant do just one thing of all those..sorry..it's not DRY.You'll be repeating something you'd specify just once.You'll have to give the system the same information in the "Register" form, and in the "Log in" form.

Am i saying, then, the Login field should be a class? Yes.It's a class.More than that: it's a complete MVC system.It has storage, manipulation, and display properties.

Then, what about the other usual fields we can find in an "User" table...like "Name", "Surname", "Photo", "City".. Yep.All those are classes too.
And you want them to be classes.You'll be using those just all the time in your life as a web developer, so you want them to be classes and reuse them in your next projects.

But still, i dont want to call "classes" to those entities, as those are, from your application point of view, just "data".That's why i prefer to call them "Data types".Because in the WWW, a "Login", is a basic data type, in the same way an int is a basic data type for PHP.

Yes, i guess it's a bit boring to write classes for all those types, specially when your schema gets bigger.But,fortunately, most of those types can be described easily:

<?php
     $datatypes["Login"]=array(
              "EXTENDS"=>"String",
              "MINLENGTH"=>5,
              "MAXLENGTH"=>15,
              "REGEX"=>"/^[a-zA-Z0-9]{5,15}$/",
              "DEFAULT_INPUT"=>"text",
              "DEFAULT_LABEL"=>"Login",              
              "DEFAULT_ACTION"=>"http://..../?viewProfile=%value%")
              /*...*/);

Yes, i accept i'm repeating myself here.The Regex contains information already specified in the MINLENGTH and MAXLENGTH fields.But it's not worth the effort to avoid that repetition (a possible way would be some sort of syntax like "/^[a-zA-Z0-9]{###MINLENGTH###,###MAXLENGTH###}$/" , but, again, not worth the effort.

You may miss some information i mentioned before, like if this field is required or not.But that kind of restrictions are not imposed by the type itself.Also, there are many more fields you could add here.For example, a default value.Or values considered to be "NULL".For example, when using dates, you may choose to consider "1970-01-01" as a NULL value.

Of course, certain "abilities" of a data type, like validation, may require more than just an array to work.Then, it's as simple as creating a class for it, with a custom validation method, and pass the description array to its base class.

So, i guess it'd be a good idea to resurrect data types.Even if they're modelled as classes, "Login","Address",and so, really are, in nature, basic data types, more abstract than "string","int" or "long".

And types have more uses than just being the basic data types of the system Model.That'll be explored in the next post!

An alternative to traditional MVC architectures for web development

In the previous post, i pointed some incoherencies about the "traditional" way frameworks apply MVC to web development.
Now, i'll propose a slightly different architecture, hoping it'll be more natural, while solving the incoherencies of plain MVC.I'll call it "Stacked MVC", but i suppose somebody else, in some place, has already described it, and called it in some other way..Duh!..

What it's shown there are 2 MVC systems.One of them, is strictly managing the bussiness side of the system.If it's an art gallery, it manages all CUD operations, like adding an artist, adding a picture, or a comment to a picture,etc.This MVC subsystem should not know anything about HTTP variables like $_POST,$_GET,$_SERVER,$_SESSION...
The controller should be invoked with User, Request, and the parameters required to serve that request.Ideally, the existence of those parameters, the sanitization,etc, has already been done.
The "V" of this sub-system, is nothing more than datasources, in a fixed format.For example, an "artist info" view, would return this info as an array, an xml, or any other format suitable to store data.
It may have it's own, private database, storing only bussiness-related data.

What invokes those "views"? The Web subsystem.
Most frameworks try to push down to the View Layer anything that is related to HTML.But, once there, it becomes obvious that the problem is not just generate HTML.A website is more complex than just HTML.There are headers, footers, metas,sidebars, client-side scripts, css files, language/i18n files..

And somehow, all that must fit in the View layer, that, having so much responsability, must use concepts completely extraneous to MVC like 'layouts', 'helpers','snippets' and all that kind of..."scripts", to finally be able to render a page.

When it should be obvious, that what it's called "layouts", is a "controller".A "Page", a "Section",a "Header",a "Footer", a "Sidebar", are objects that exist in the context of a website, so they're Models.A web page is much more than simply a bunch of HTML.There's a logic between all the components of a web page, much like there are relations between artists and their pictures.
To render bussiness subsystem data, the website subsystem uses the previously mentioned datasources, the same way it uses a database;The "model" objects in the website subsystem ("Section"), "queries" the bussiness subsystem to fetch the data it requires, and sends it to a certain "Section View".

Any "R" request arriving at the web server, is sent to the website subsystem.
Any "CUD" request, is sent to the bussiness subsystem, and, if that request is configured to generate a "R" request after the bussiness subsystem has processed it (onError, onSuccess), adds to the famous tuple of $_POST and $_GET, information about the result of the processing done.

This is just a quick intro to this idea.There are more advantages, gotchas, and ideas around it.But, to finish this intro, i'd like to say that i'm not trying to say we need MORE frameworks (O-M-G)..This architecture is more about having a guideline on how to distribute the application code, than using (insert bloated framework name here) to solve it all.

Is MVC a good paradigm for web development?

It's like you'll get laughed at , if you dare doubting about it.Checking Wikipedia article on MVC you'll see the *myriad* of php MVC frameworks out there nowadays, so, looks like it's definitely the way to go.
But, still, i miss some kind of reasoning of why MVC is such a great paradigm for web development.Sometimes, i get the impression it has been adopted because one was desperately needed.
Of course, there are plenty of sources explaining what MVC is all about.Let's take the definition of "View", according to Wikipedia:

A view queries the model in order to generate an appropriate user interface (for example the view lists the shopping cart's contents). The view gets its own data from the model. In some implementations, the controller may issue a general instruction to the view to render itself. In others, the view is automatically notified by the model of changes in state that require a screen update

Now, let's look at the definition of "View" we can find in CakePHP (taken as example):

Views are the V in MVC. Views are responsible for generating the specific output required for the request.Often this is in the form of HTML, XML, or JSON, but streaming files and creating PDF’s that users can download are also responsibilities of the View Layer

Are those two definitions equivalent?For starters, the second definition includes a new concept, that looks like it's not important enough to have its own initial in the paradigm name: the Request.
But, if we continue reading CakePHP documentation on Views, we'll find quite a few more concepts:

The view layer in CakePHP can be made up of a number of different parts. Each part has different uses, and will be covered in this chapter:

• views: Views are the part of the page that is unique to the action being run. They form the meat of your application’s response.
• elements: smaller, reusable bits of view code. Elements are usually rendered inside of views.
• layouts: view files that contain presentational code that is found wrapping many interfaces in your application. Most views are rendered inside of a layout.
• helpers:these classes encapsulate view logic that is needed in many places in the view layer.

So, the view layer, are composed of a few things, one of them being the "views".But the "views", are a "part of the page".Another new concept that is brought up.So, "views" are part of the "view layer", and the "page".Are those two things the same?
Because "layouts" are defined as "view files" that wraps "interfaces".Again, "views" are rendered "inside" a layout, so i guess it's safe to say that views are also part of "layouts".
So, "views" are something that are part of the "view layer", the "page", and "the layouts".
It's worth noting that "page" is a HTML concept.There's no such thing as an XML or JSON "page", and i dont think "streaming files" can be thought as a "page" neither.
Going back to the MVC definition found in Wikipedia, a view "generates an user interface".It could be argued if generating HTML is exactly the same than generating an user interface.But, i dont think XML, JSON or "streaming files" are exactly "user interfaces".

To finish the soup of concepts, while "elements" are "reusable bits of view code" (they are view code, but they're not views?), "helpers" encapsulate "view logic needed in many places".I swear i could write as "reusable bits of view code" as well.
I'll not go into the "view blocks", another concept introduced later in the same documentation.I think it's enough. We have the "view layer","view blocks","view logic","view code","view files" and plain "views".
This way of abusing the MVC definition, using the word "view" repeatedly, only shows how poorly defined are each concept, and, finally, how the documentation tries to twist concepts to make it fit into the MVC paradigm.
Let me demonstrate that: if i pack a certain user information inside a JSON message, what role takes that JSON message in my system? Is it an "user interface", like the "canonical" MVC definition refers to what is produced by the "views", or is it the way a Model residing in the client, is getting its data, the same way the Models residing in the server get their data from the DB?
Is the "V" doing the same job for our system when it sends HTML (data mixed with presentation), and when it sends JSON (just data)? Can you think in a system where the server does *nothing* about the user interface? (Flash? Desktop / Mobile App? Javascript?).Then, how can we call that a "View"?  Whats a "page" in Flash terminology?
This is why, in the CakePHP docs, they've subtly changed a word from the V definition at Wikipedia. Views no longer generate "user interfaces", but "output".Any "output".Way to generalize an idea to finally push inside absolutely unrelated concepts (as if all "output" has the same role in a system..).

You may wonder if i'm attacking CakePHP for some reason.Not at all.In fact, the first time i've visited CakePHP site, has been for writing this post.I know beforehand most MVC frameworks docs will be extremely vague, inconsistent in this point.Because, simply put, it doesnt fit well.But doc writers really try hard.And most people will read those docs trying to learn, not trying to critisize it.I'd have pick any other framework, and i bet the same inconsistencies would show up.

Those are some of the reasons because i'm not sold on the MVC fever.And, also, because i think there are better, more consistent ways to do web development.

PD: Just checked Symfony documentation.In a whole chapter "explaining" Views (like 15 pages), "model" and "controller" are mentioned a whopping total of..4 times! 3 of those times, in side-notes.All the documentation focus on all those little "helpers","elements","layouts" (read: good old scripts)...But nothing about how the View / Model / Controller interact.

Describing a web page (Describing vs Coding II)

The most common concept in web development...is a web page.
Is possible to describe a web page?
A simple try: lets try to describe a web page where you can see info of a certain artist, along with artwork he uploaded to our site:




<?php
       $webpages["viewGallery"]=array(
             
             "PERMISSIONS"=>array("ANONYMOUS"),
             "RENDERER"   =>"HTML",
             "HTML"       =>array(
                     "HEADER" => "default_header.html",
                     "FOOTER" => "default_footer.html",
                     "RESOURCES"=>array(
                                "CSS"     =>array("viewGallery.css"),
                                "SCRIPTS" =>array("viewGallery.js")
                               )
             ),             
             "PARAMS"=>array(
                              "id"=>array(
                                           "MODEL"    =>"Artist",
                                           "FIELD"    =>"artist_id"
                                           "REQUIRED" =>true,
                                           "SOURCE"   =>"GET"
                                          )
                             ),
             "ROLE"  =>"VIEW",
             "MODEL" =>"Artist",
             "VIEWS"=>array(
                            "ROLE"    =>"COMPOSITEVIEW",
                            "LAYOUT"  =>"STACK",
                            "SUBVIEWS"=>array(
                                array(
                                      "ROLE"    =>"VIEW",
                                      "OBJECT"  =>"Artist",
                                      "VIEW"    =>"Bio"
                                      "PARAMS"  =>array("artist_id"=>"id")
                                     ),
                                 array(
                                       "ROLE"   =>"LIST",
                                       "OBJECT" =>"Artwork",
                                       "VIEW"   =>"artworkList",
                                       "PARAMS" =>array("artist_id"=>"id")
                                     )
                            )
                          ),
       "CACHE"=>array(
                             "TYPE"            =>"TIMED",
                             "INVALIDATE_AFTER"=>30
                      )
        );
   

This array may look complex, but i invite you to read it.Again, bear in mind that this is just a "sketch" of what i want my page to be.It's not related to any particular framework.It just tries to express concepts.

But, what i consider interesting, is that many things that usually are coded, here are described.
Most pages will have a header, and a footer.Many, will require permissions.Lots will use parameters, and will have to sanitize them.Will include external resources. Will include one or some other pages.Other things that may include..headers (encodings,etc), tags...May require a certain "state" to be set.May set a "state".. Things that are easily described, but that usually, are just scattered around in code (if not forgotten).

When you are writing the above description, you're focusing in what you need to create the page.Not in how you'll create it.This is key.
And, as there isn't any kind of formal structure, you can add whatever keys/values that may fit your needings.

If somebody has to maintain my code, and reads the description array, he'll have lots of information that will help him to understand the actual code.Of course, as you may begin to notice, most of that code, could / should interpret that array, or will be generated from it.

And,if someday i want to switch from one framework, or language, even programming paradigm, having lots of information about my system as *data*, instead written in code, will ease the transition.

And this is the main concept; i don't know if there are frameworks using similar ideas, but, that's already a contradiction;my idea is: freely describe your system as data, without any dependency , syntax o requirements a certain framework may impose.
I've written an ORM that i (later) found to be pretty similar to Django for python.But, the fact that model objects definitions are arrays without any dependency on that ORM, is what makes both completely different in philosophy, and it's the philosophy (describe your system, don't just code it) what i consider important.

The challenge is to find the right way to describe a web page may have, from models to db-mapping, views, pages,permissions, in a way that it's, at the same time, intuitive, extensive (covers as much of the functionality as possible), inter-related (so form definitions rely on model definitions), and external dependency-free.

Describing vs Coding

I like to start the development process just doodling some circles and lines in a blank paper page.I dont use formal symbols..and almost always, after two days, i really cant decypher what i was trying to express in it.

I think doodling is something many of us programmers do.What it's interesting, is that those doodles are the first bits of information about the system.Yes, very rough, too general, and too simplistic.But still, "information".

For me, this raises the question "The description of a system, is data, or code?".And, "If it's data, where is it?".The doodles are some sort of strange data.But, could it be possible to save that data in some way that it's usable for actually develop the system?

There are other ways to express the problem.If someone describes a website as having a menu, with 4 sections, one of them having a form where you can enter a comment, i bet that, with just that, you can already imagine a lot about that website.Because, implictly, the words "menu","section","form","comment" already have a lot of meaning.

Is it possible to actually start development with such information?

Ok, let's suppose we start developing the website, creating this file:




<?php
         $myWebsite=array(
                        "TITLE"=>"MyWebSite",
                        "SECTIONS"=>array("Home"=>"BLOG",
                                          "My Gallery"=>"GALLERY",
                                          "About"=>"PLAIN"),
                        "ALLOW_USER_REGISTRATION"=>true
                        );
         ?>
   

That would be a "doodle" written in php.I still don't know what the keys and values of that array really mean, or how they will be processed.But, i bet that, just reading that array, you may have an idea of what kind of website it describes.And that idea contains much more information than what it's strictly specified in that array.Intuition is being translated as "reasonable defaults" in computer programming.

Now, suppose you drop that file in the top folder of your project.Deeper folders will add finer detail.

Suppose we create then a "Sections" folder, and add the following file, Sections.php:



<?php
/*....*/
$sections["My Gallery"]=array(
     "GALLERY"=>array(
     "VIEW"=>"PictureList",
     "ITEMS"=>20,
     "OBJECT"=>"Picture",
     "SINGLE_ITEM_VIEW"=>array("VIEW"=>"PictureDetail", 
                               "ALLOW_COMMENTS"=>true),
     "HEADER"=>"DEFAULT",
     "FOOTER"=>"DEFAULT");
     /*.....*/
    

In the first file, it's specified that "My Gallery" is a section that follows a "GALLERY" pattern, whatever is that.It's still a vague notion, but it's safe to say it's some sort of listing.In this second file, we parametrize the pattern, with, again, more vague details.

In this file, i try to express that this gallery shows 20 Picture objects per page, and will use the "PictureList" view to render them.Also, if i choose to see a single item, i'll use the "PictureDetail" view to render it, and i allow adding comments to that item.

Still, i don't know how i'll handle all those "VIEW","ITEMS","OBJECT" concepts.I haven't decided what framework i'll use, what kind of database, if those arrays will be loaded each request, or used to generate code.I'm still describing the system, instead coding it.Iteratively, adding information about what the system is, in a format that is easy to manipulate and transform, so they really are *used*, they're not simply doodles in a paper.

Of course, the above examples are not really good examples for describing a website. In fact, it may be hard to express things in a php array (or xml, text..whatever) that intuitively you can see clearly in your mind.

I've been using this kind of approach for 5 years now, and use it almost anywhere, from model object definitions, to views, forms, pages,states,storage...Sometimes, the first time programmers in my team see those arrays, they are sort of surprised.But my experience is, once they understand the core of it (describe as much as you can, and code less), they seem to be happy with this "paradigm".