And then I had an epiphany.

There is no need to create a tool to generate configuration files. I created four tables (as a proof of concept) on MySQL, one for the tables, one for the fields, one for relationships, and one for validation rules. By creating a table configurations file with a PHP array that connects all those tables properly, it becomes possible to create new configuration files using the data handling library itself – since the configuration data is on tables on a database, it is possible to manage it using the generated HTML pages, or actual code. When the user is done, it is possible to select a table and all its fields, and use the export() function to export that data into the php array format that is used by the data handling library.

Formidably, this all is merely an intelligent use of the library, and nothing new is necessary to make it work. The generated configuration files are PHP arrays, which means there is no extra overhead (as is the case with INI files, XML files, or even getting the configurations from the database), and they can be imported/exported on demand whenever necessary, so if anyone should ever port this library to another language, they change their “configfile” export implementation to match their language’s most efficient data structure.

But the idea of having to export configuration data into files, and having to have a database for managing configuration data still seemed too much for me. I even considered using a self-contained SQLite database, but that would still force users to export data into configuration files. I could make it so that it would use the SQLite database rather than relying on php files and arrays, but that would mean overhead that I wasn’t willing to accept. So, I came up with something else:

The database containing the four tables will be in the format of a PHP array itself, (which is possible thanks to how easy I’ve made it to create new database drivers), which removes the need for exporting completely. The php configuration files will instead be edited directly. By default, php arrays will be used, but any other data driver can be used, if you wish (which means you can use YAML, XML, or even a real database system, such as MySQL or SQLite). Included in the distribution of this library as well, there will be a php script that can be run on the browser to effortlessly start managing table configurations.

A few additional methods are included in this library to make it even easier to get started: discoverTableStructure(), discoverDatabaseStructure(), createTableStructure() and createDatabaseStructure().

The discover structure methods simply look up a table or a database’s structure and generate a configuration file from it, as well as adding it to the SQLite database for easy management.

The create structure methods take an existing configuration file and automatically create the table structure represented by them. This helps importing database structures onto a variety of database systems, very easily.

updateDatabaseStructure()

This is actually one really big problem for most developers: How to keep the database structure up to date at all times. There are a few projects for keeping a database under a version control system, the best of which is (in my opinion) dbdeploy. They use a table to keep track of what updates have been applied to any given database, and the so-called delta files to figure out what SQL to execute for a certain update, and how to roll it back.

I wanted to take this step a bit further. Imagine opening up your users table in the database admin, adding an extra field called BIRTHDATE, checking the box that says “Do you want to update the actual database structure?”, pressing enter, and knowing that that’s the last thing you’ll need to worry about? This involves a bit of set up on the repository’s side – you have to tell it to ignore the dbversion file, because that’s what contains the current database version, and you need to set it up to automatically run updateDatabaseStructure() upon deployment. These things can still be done manually, but my goal is to make it as effortless as possible, and if we can use the version control system to help us out, that’s even better.

And if you ever make any changes to the database structure that require you to run some SQL to keep it all working, you can add that in along with the update (but you’ll be required to write it in code so that the library can generate the SQL for all the different database systems and generate rollbacks as well).

Well, that’s one thing out of the way – the database structure. That’s extremely important for me, because it’s the foundation of the whole library. I know that most database libraries don’t rely so much on configuration files describing the database structure, field validation rules, relationships, and all of that,  but this isn’t merely a database library, and it is this reliance in these configurations that allows me to make it even easier for people to use the library, and even manage their databases. With this library, you will never have to worry about anything related to your data again. Sheer simplicity is on its way.

So I set out to think about what such an application would look like, and how it would behave. For one, I will be creating a complete set-up helper, to guide newcomers and instruct them both on GTD and the application itself. I don’t want to make it too complex, though, so I will stick to the minimum necessary to give them a good understanding of GTD, and if they want to learn more, they can always buy David Allen’s book. This guide would show up on the first log in, and there would be two other routes that a user could take: “I know about GTD, just tell me what I need to know about this app” or “I know about GTD and I’m sure I can find my way around here, the app isn’t that complicated anyway. Go away and leave me alone, helper.” (I believe that their description is self-explanatory )

One of the things I’d like to have is a virtual personal assistant. Since GTD consists of dumping all of your mind’s contents into a place that you can trust and that you will constantly visit, I thought that it would be nice if I was able to just log in from anywhere, and have the app tell me what I should do, how I’m doing in terms of productivity today, and all of that. But this virtual personal assistant wouldn’t have to be that simplistic. Ideally, I would like it to be able to manage everything I have to do, and guide me, in the most intuitive way possible. I’ll develop further on that later.

I think that for this particular application (and considering that there are so many productivity applications out there), integration with other applications is vital for its success. For example, GTD requires the use of a calendar. The application will have a pretty powerful calendar built-in to handle everything, but I think that most people would fall into one of three situations:

1. They’re used to a certain calendar application and don’t want to change.
2. They would be okay with changing calendars, but it’s just too much effort to transfer everything over, plus they don’t know if all the features they have would still be available.
3. They are okay with changing, or they don’t really use a calendar application at the moment.

So obviously, for this application, its calendar must work with other calendars, and make it as simple as possible to keep the calendars in sync with each other. I want to do everything to make this application the easiest to use and get used to possible.

One of the most important things to become more efficient, is to reduce the number of inboxes as much as possible, so that we have less places to check for new items. With the internet, it seems that our number of inboxes is increasing exponentially, with every new web service opening up a new one. Thankfully, however, most of these web services have APIs, and with that in mind, I want to make it possible not only to bring everything together into one main inbox (this application’s inbox), but also to deal with it in the application. Imagine this central inbox containing new e-mails, new @mentions to your Twitter account, new Basecamp tasks/projects assigned to you, or maybe even Facebook direct messages sent to you. I want to make it so that you can log in into this app, and everything you need to do or deal with will be sitting there, waiting for you. Because I am tired of having to check a bunch of different places. If I could read an @mention and reply back on this application, read an e-mail and reply back, or even read a reply to a blog comment I left somewhere, and reply, all within this application, that would make my life far, far easier. The less effort it takes to collect, process and deal with inbox items, the better it is, and the more efficient the application will make us all.

That’s it for now, I’ll follow up soon with some more ideas.

The main features/ideas for this library so far:

• Create, read, update and delete records - as is expected from modern database abstraction layers. It only works with MySQL at the moment, I am planning on using the Adapter design pattern to make it work perfectly with other databases.
• Validate data - At the moment, the validation system is a bunch of cases in a switch system, but I intend to make it much more powerful.
• Filter, sanitize and escape data – so that you can be sure your data is always valid, and won’t break your website. At the moment, not much work has been put into this part, I’ve been working on data relationships, but I am thinking of making it very strict, allowing nothing by default, so that developers are compelled to display their data properly.
• Analyze data - Be it simply counting all the rows in a set of data, finding out an average, or performing more advanced statistical analysis, I want it to be possible using this library.
• Import/Export data - I want to be able to convert data between different formats and import or export that data into different formats. For example, I’d like to getLatest(10)->export(‘RSS’) and get an RSS file containing the 10 latest results in a data set (getLatest would know which field the creation date field is, and act accordingly). JSON, CSV and XLS are three more data formats that I’d like to support in the near future.
• Generate the HTML – Necessary for accessing and manipulating the data, as well as repopulating the forms when validation fails, inserting default values, etc.. This has to be extremely flexible: it must be possible to integrate this into an existing theme/layout as easily as possible, and also to make custom layouts, in case the developer wants to.
• Handle Data Relationships – This is extremely important to me. For example, if I had an instance of this library that had all the information about my users table, I would like to not only be able to handle the data on that table, but also get all the posts related to that user, or some additional user metadata that’s in another table (WordPress does that, for example). This is possible at the moment using custom fields (more below).
• Custom Fields – Sometimes, you don’t just want to stick with the fields in your database table, you want more. For example, you might want to display a list of your users, and the number of purchases they have completed on your website. Or you might wanna get an array of IDs of posts that your user has written. With custom fields, you can – you tell the library where the data is, and it will take care of everything for you.
• Field Identifiers – Field identifiers allow you to completely abstract your code from the underlying database structure. For example, let’s say you have two database tables, one with USERNAME and PASSWORD and the other with user_name and user_password. Or you had USERNAME and PASSWORD, but decided to change them to something else. By using field identifiers, it is possible for you to change the database fields, and the only thing you have to update is the configuration of your data set. The code remains the same.
• Method Query Language (MQL) – While developing this library, I felt that I needed a way to make my querying even easier. I had found myself writing functions getById, getByUser, and others, over the years, and I decided to implement this into the library. The results is an astonishingly simple way to both query and update the database: getFieldsByFields() and setFieldsByFields(). An example would be getUsernameById($Id), to get a user’s username, given their ID, or setAddressAndZipcodeById($Id, $Address, $Zipcode) to update the address and zipcode of the user with a given ID. It’s that simple. And thanks to Field Identifiers, if your database structure changes, you still won’t have to change the function names, so it won’t be a problem.
• REST-ful API – Just about every modern day website has to have an API to enable remote access to their data. The library includes an api/ folder, with everything necessary to make remote calls to data using method query language. By default, you can use: http://www.example.com/api/api-key/data-set/method-query/arguments. An example of this is http://api.example.com/api-key/articles/getLatest/3. This API can easily be customized to your needs, and by default it is impossible to access or use any field in the API, you have to enable API access for them manually. For example, you might want to allow API access to a list of your posts, their author’s, post creation date, and all of that, but not their metadata.

Those are the main things I’m working on at the moment. I’ll be discussing them further here while I develop them, and I would like to hear your feedback. I will also release the code under the MIT license very soon, maybe even set up a git repository or something, in case any of you wants to help me out developing this mega project.

It’s easy to use this library. Basically, you create a configuration file that contains the fields in a database table, their validation rules, permissions (whether or not you can edit that field in the HTML forms, whether or not you can access the field using the API), and all of the things pertaining to the database table. Once that’s done, all you need to do is instantiate the library, pass it that configuration file, and you can start using it straightaway. Simple.