Wednesday, November 2, 2011

d-jon is going to the open source world!

After 1 year of hard work I decided to open the code to the world, why? easy because I always wanted to give something to the community, I used several tools based on open source and I think it's good to give something back.

How d-jon started?


2 years ago I was working on the java version of Bizagi (http://www.bizagi.com) and the team started to use TSP (Team Software Process) in order to develop the new rendering module, one of the key features of TSP is the ability to measure the time spend on every single task you do on your job, that's how I started to use several linux tools to track my time and all of them were unsuccessful, most of the tools because they're unable to track if I'm working on my desk or leave the office soon, if I forgot to stop the timer when I decide to take my lunch then I had to get back and pinpoint at what time I leave the office, same happens when my boss (or a coworker) ask me a question about some other subject, when I come back to my desk I did not know how much time I spend assisting somebody else, one of the tools lost all my records without any warning, etc etc... I got tired of this and decided to create a simple tool to track my time.

Some months later I realized that my tool was getting stronger, and stronger, so I decided it worth the effort to put d-jon on the web, that's how I did the first refactor of all the code to create a less buggy application, at first I was the only user so I can deal with some bugs, but now that the application is going to be on web I have to fix them.

Vision Software, the company I have been working for 7 years, decided that this tool was good enough to replace the Allnetic solution that was buggy and has several issues. 1 year later d-jon has over 10.000 downloads and it is been used in several Countries, I'm pretty happy that it filled the gap between professional solutions and the free ones.

Why going open?


First of all, one of the most difficult tasks is to get some feedback from the users, not because they don't want, just because there is a lot of spammers out there willing to put porn and viagra ads in every site they can, I started a forum and tried to keep the spammers out of it but was a futile war, they won... crash and burn me, I do not have the time to deal with web issues, and be a policeman of every post on my site, that's why I closed the forum, I had to spend too much time deleting spam messages, then figuring out how to keep the spammers out, etc... I prefer to use that time to improve the application and not to keep these unwanted people out of my life,

So, what if the users do the extensions and fit the tool to their expectations? that's why I decided to open the source code, to let the users improve it and get the best from the tool.

What's comming?


I need to do several things before publishing the code, like license to be used, put all the documentation about the code, create a standard structrure (make and all of that stuff), etc etc... but I thing it worth the effort.


Sunday, October 23, 2011

My own NoSQL DB, progress


The last few weeks have been good... hard... but good, I finally implemented some new layers in the DB, some of them were harder than others, and i realized that I dont know that much of some bare metal components like network layers as I thought I know, but anyway I managed to pass through all of this and the db is now running smoothly, this post is to write down some of the progress made and explain some of the problems I had implementing these new layers.

Command Layer 

This component is really one of the cores of the communications with the user, basically it's meant to process the commands made by an external player and figures out what is the request, how to handle it and what is expected to be returned to the client.

This component was not really a problem, basically because I already knew how to handle the network protocol, then I wrote a writer and a reader and that's it, the command is alive!.

Network Layer

This component was really a hard ball, I didn't know that synchronize the sends with the reads will be that hard, this component drove me crazy for some days, until I realized that I had the network layer does not sign a contract with you to send exactly what you put on the pipe, thus is not suppose to get the expected length at the end of it. Let me put it this simple:

Client A writes 1000 bytes on the pipe.

Server expects 1000 bytes...

But... that does not happened, the client actually sends some chunks of data, 100 bytes at first, 500 later, 200... etc. Thus the server should expect to read this mess in the same way, now that I read some literature about it (that's why it took me 1 week to implement this layer) I know that it's easy... but this layer was a real pain (if you have questions about this, just let me know and I will try to explain how to create a reliable server/client communication), I still to do some improvements in order to get better performance, but anyway it's working right now and I dont want further delays in this project.

Index implementation

Here's the tricky part of the databases, figure out how to create an efficient way to sort the data in order to retrieve ii later, in this part I tried to avoid the conventional methods, and try to figure it out from my own knowledge... big mistake!... hahahaha, anyway this tries outs were good to understand why the people came out with fancy solutions like B+Trees, BTrees, etc. here's the BUT... I read documentation on B+trees and it didnt fit well on what I think should be the separation between data and indexes, so I had to create my own solution to this problem, of course... based on B+trees knowledge and some of my tries outs.

Now, the tests runs very well... the db gets the following output:

20000 inserts per secs and 29978 finds by key operations per sec.

The thing with the network layer is not going that well, but I will fix this later, with the network layer on the output is: 8695 ops, close to the achieved output of MongoDB which makes me very happy for now. I know that when the time comes I will fix the network issues and that will be a nice boost for my server.

Later I will create some posts of my learning process and what problems I had to solve in order to keep on with this project.

The project keeps looking good, and I will keep going on.

Thursday, September 22, 2011

My own NoSQL DB

15 days ago I read this article: http://blog.cubrid.org/dev-platform/nosql-benchmarking/ and got shock with the amazing performance, c'mmon! 21k operations per second? it's amazing...

After a long bath I started to wonder, how much performance will I achieve if I create my own NoSQL database, just an experiment for those rainy days, ok... here I am 1 week later after some hard work, I created my own NoSQL Database which is capable of:

1. Insert Records
2. Find records using the key (that's the benchmark)
3. Index the keys and whatever I want to index to find records later.
4. Split files to get indexes separated from data.
5. Created a layer to save the data using little endian (basically I copied the code from my other project d-jon)

Here's the test I did, with a modest machine and 8Gb of RAM:

1. 1M inserts of 1k (I know, the bench was over 50M, but I dont have free space on my machine right now)
2. A random search using random keys (300.662 searches)

And here are the stats:

the inserts took 23.48 secs, which means 42.584 operations per sec.
The searches took 20.3 secs, this means 14.763 operations per sec.

These results are preliminary, but I'm starting to think that I could beat MongoDB, Cassandra, and others,

I started my project because I was out of challenges, and it's turning to be a very attractive project, maybe this could be my next fulltime job.

A lot of work coming, but what the hell... djon started just like that and now have over a 3k of downloads and users all over the world.

Wednesday, August 17, 2011

C++11 C++x0 new C++ is here!

At last, after almost 3 years the new standard has become a reality, this new standard has a lot of cool features that I will try to cover within my next posts.

Follow the link for The C++ Standards Committee if you want to get a "light" read and have an understanding of what's going on with C++11. Or just keep connected because I will try to explain the major changes about the standard.

Thursday, April 21, 2011

Almost completed

Ok, my game is almost complete, at least the main idea, interaction and elements required (graphics, sounds, gameplay, etc), during the last 3 hours I created the level system, an explosion animation (I didn't find an animation that suites my needs so I need to create it from scratch using gimp and imagination), I also added sounds.

In Addition, my tester (my 4 years old son) did a test and I think it worked as expected, everytime he see me on the computer ask to play the game again and again... the magic is done!

Now I need to work on the following things:

1. The health of the UFO
2. The health of the player
3. The fight back from the UFO (easy with the engine I created)
4. An score system.
5. More sounds.

At this rate I think I will finish my first game in 1 week (or less if I have time during my vacations).

I'm very happy with the current results and I will share some demos of the code (not the one used in the game, but I will manage to create some demos of how I used SDL in my first game).

Tuesday, April 19, 2011

Week 2 Results

Then game is going pretty well and Im starting to love this new career, the results are very good (from my limited and biased point of view) and I want to share the results that I got till now,

Game Engine:

The main idea of a game engine is to create something that you could reuse easily on other projects and avoid the creation of the code from scratch everytime, my 2d game engine has the following features:

  • Image loads (maps, sprites, etc)
  • Movement (gravity and simple movement from left to right)
  • Event managed in a proper way
  • main loop routine
  • Collision detection

So far this is a very good result for the second week of work. But I didn't start this from scratch, the www.sdltutorials.com was a very good help in order to understand where I needed to start from.

Game Content:

This was a little bit difficult, I changed my mind several times about how the game will be, cause everytime you see the screen you want something different, it's hard to keep the original idea because you see how the gameplay will be and then you will change the way you thought it's going to work at the first place, but anyway... that's the idea, improve, adapt and get the best from the game.

Most of the spend time was trying to figure out how to create the images you need to put on screen, I tried to create them from scratch but was a big failure, that's why I decided to google a little bit and found some images that I could modify to create the background of my game, also the ufo was taken from a picture found at google, removed the background and get the ufo out of the picture, the cannon was harder but I found a nice place with tons of sprites that I could use (I will post his link in here later), the missil was created from scratch using inkscape (very good tool).

The mathematics was a little bit tricky but finally I managed to create the cannon effect of following the ufo and also the missil will goes into the direction of the UFO's center.

The results of the second week were saved in a video



Hope you liked the progress and if you have any questions I will be glad to answer them.

Shoot the missil: Maths in action!

I came across a problem these days, how can I send a bullet right into the direction of another target, here's when you need to get back into the school and use your Pythagorean theorem knowlegde,

After spending some hours trying to solve a simple equation I realized that I've to get back to the math books, I found a very good discussion about this subject in yahoo answers what you need to know about maths in order to create games.

Ok, I wasted 2 hours into this equations and then remembered something (very blur btw) about vectors, how they should be used to move a point into the direction of a line (vector), that's how I crossed with this: http://www.gamedev.net/topic/600086-find-point-between-2-points/ actually the problem was very easy to solve:


p1x=1;
p1y=1;
p2x=4;
p2y=4;

dirx=p2x-p1x;
diry=p2y-p1y;

p3x = p1x+dirx*0.25;
p3y = p1y+diry*0.25; 

First you need to find the direction of the vector, you get that doing a substraction between the 2 points (x2, y2) - (x1, y1), now that you have the direction you should apply it to the new point and that's it. easy right?

Now my game has a beatiful rocket moving into the direction of the UFO, one step further in the game creation.

Thursday, April 14, 2011

Week 2: The engine is ready and I'm working on the graphics

The last week was amazing, I thought was going to be harder to create a game, but thanks to sdltutorials.com the engine was created in 2 days, and I started to work on the character interactions.

The first thing I noticed is that the game is not going to be what I started with, actually is very different from the sketch I created 1 week ago, that's because everytime you see the screen working you change your mind and want something new, at this moment the game is going to keep it's scent but several details has been changed from the original idea (things like how the cannon will shoot, when will be the user dead, etc etc).

After the plot of the game was created I started to learn I little bit about sprites, and how they worked, this is very easy subject, but the animation... that was a very different history, I tried to create my own sprites and was a great failure, that's why I started to look for some free sprites in order to avoid this work.

This is how the screen looks like at this moment:


The cannon follows the ufo and the ufos moves as expected. I had to remember how the angles worked and what is all about cos, sin, tan etc that I saw at school and put under the bed a very long time ago, I'm realizing that I need to study very hard mathematics, my brain has some "rusty" at this moment, but anyway I will need to get my mathematics book and put in shape again.

My accomplishments this week:

1. Engine created
2. Basic graphics created
3. Basic movement of the UFO and cannon working

Next week...

1. Cannon shooting
2. UFO crashing and burning

The main idea of the graphics are settled, and I think I will post a video on youtube with the current animation that way you will give an idea of how the animations are, anyway as always I will keep the game idea hidden.

Tuesday, April 5, 2011

Learning SDL

I'm really impressed about how easy is to use the SDL library in order to create your games, I followed the tutorials Basic, Coordinates and Blitting, events and the tic tac toe sample from http://www.sdltutorials.com and it's really simple and straighforward, actually I though it will be harder, but the explanation was amazing an I understood everything in minutes, the examples ran very well and I'm really thinking that this is piece of cake, the game is on its way to sucess.

Here's the result:


I will finish all the tutorials of sdltutorials and then I will start the game I'm planning, but I think everything will be set when the animation tutorial finishes (and that will be tomorrow), I will spend sometime understanding how the collision system works but I will not need it in my first game.

After this tutorials the next step will be the graphics, that will be easier because I've some experience using graphics tools, and I dont think will face any problems in this area.

Step 2: API decision OpenGL or DirectX

This step is a little hard, I need to choose what will be the API I will use to create my first game, in the future I will learn both but I need to start with something, this time I will focus my decision on the following principles:

1. I want to learn how to work with games (both will be handy for this)
2. The game is fairly easy and will not require tons of hardware/software to run, so I don't care which one is faster/better than the other.
3. I usually in love with the open source/freedom and dont like to be pressured to use something
4. The game will run mainly on desktop machines (Windows, Linux or Mac), some of the games will use the keyboard/mouse to run.

After reading this article http://blog.wolfire.com/2010/01/Why-you-should-use-OpenGL-and-not-DirectX I decided that my first API will be OpenGL, and I will try to see how to use the MVC pattern in games in order to abstract the API usage, if I can achieve to run the game using directx and opengl will be great, but my first game will be as easy as it could, that's why I will not shoot myselft if I cannot manage to run it on directx too.

Ok, decision made, now is time to read a lot of OpenGL, to my advantage I have some knowledge about Qt so I could use the Qt-OpenGl interface and speed up the process, let's see how it works.

Monday, April 4, 2011

Step 1: The game plot

Here's the easy part, you should figure out a game idea and put it on paper, that's my first step in this journey.

In order to avoid any further problems I will say little about the game, instead I will put some of the key points of the things you should list in order to start the game:

1. What is the audience of your game? in my case I will create a game for kids between 4 and 5 years, why? because my boy has that age, and the games for kids are a little bit easy, catchy and does not required full 3D render in order to catch their attention, also I think the educational games are the best, if you're going work for free then why not give something to the world?
2. What is the game for? what is the idea of the game? here's the hard part, if you want to teach something (like I want) then you need to know how to teach that, here I will base the game on some other games that does something similar, I will create an idea based on some letters games I saw sometime ago and I will put some ideas together in order to create an original game.
3. The sketch: here you will put your idea on simple graphics, could be on a paper or in the computer, that does not matter, this will help you to realize if the game will be playable and your idea makes sense. I created some simple graphics using inkscape.

Ok, here's the first part, the next post will be about the articles you should read, the exercises you will need to face in order to start with your knowledge base.

Starting with Game Development - Journey

Ok, here I'm, tired of doing the same thing over and over again, stuff that uses/saves something in a database, support a business and get the people rich... I'm tired of the same job, that's why I decided to give a little turn in my life, change and start doing the thing that put me on the development lane at the beginning... THE GAMES!

Most of us started at development when we were young and wanted to know how the games worked, we hacked some of them, changed the fixed texts to include words like "tits", "ass", etc... that was for fun, and started to realized that making games was the thing you want to do for the rest of your life... in some point something changed, you had to pay bills, and raise kids, etc... and you got stuck in the DB/related programming world.

Now that I realized that I want to be in the original path, I'd a good background in development, I'm Java Architect, had some certifications, 15+ years of programming experience in java, c++, delphi, VB6, C-sharp, etc etc etc etc... so I will start to record my progress in the game world, and now... I noticed that I don't have a clue of how to start, thats why I will do it from scratch, I'm reading some guides, and I will post some of them in here, I will record my steps on how to build games and how to start in this world. I'm starting with this so I will do a log and document how everything goes, in 1 or 2 years I will read this again and I will say: "I made it!, I'm not doing DB stuff anymore, I'm a game developer, and the people will say... Oh man, that's why I started in development, I envy your life" or maybe I could say: "It's a shame I started and now I know this is not funny at all, but at least I did it", lets see how this end.

Wednesday, March 2, 2011

Why should I track my time with d-jon?

I had the same question two years ago, why should I sacrifice myself with the task of tracking all the activities I done in a day?

The answer came to me when we started using TSP (Team Software Process) in the company I work on, I started with some resistance to the concept of tracking my time, mostly because my train of thought said:

1. If I track my time I will not have time to read the newspaper on my work and will be like a slave running with the clock every single day
2. If I start to track my time I will never be able to get out early just because I will be always late.
3. When you track your time you will have to report the time to your superior and he will ask you to finish all the tasks based on the task with less time.
4. If I track the "real" time spend on a task I will not be able to use "reserve times" (Contingency reserve or management reserves) and when the known unknowns occurs I will be working until 2 am.

etc etc... What I discovered after 6 months in this new team work was that:

1. When you record your time you have the tools to explain why you take X time into a task, and you could show your boss that you used your time well, but some unknown tasks (that I know now) are messing with your times. For example I discovered that 20% of my time was spend on activities not related to the project I was working on, for example, meetings, explanations, customer support, etc etc. With this new real times, you can predict the behavior of new projects, and create tasks that support your real work. If you don't track your time you will not have that valuable information. In addition you could reassign some tasks to somebody else in other to avoid these "not project" times affect your work.
2. In most of the new projects I don't need the reserve times, just need them when a task is really unknown, and we spend some time in proof of concepts in order to determine that time.
3. Now I can be in my house early, I can say "hey, I worked 8 hours straight and I will leave the office", before the time recording everybody saw me as "hey, why he is leaving early, did he really get into the office earlier or just worked less than the other parts of the team?"
4. If I put an estimated time in a project I've data that support why I will spend that time, no one will be able to change my times because I already know how much time I will spend on that kind of task (ok, it's true... not all the tasks are exactly the same, but now I have some idea of all the activities I should do in order to accomplish that)

What I encountered during this time is that the tools to track the time are not really good, that's why I created my own tool that I offer for free on http://d-jon.com I hope you use it and let me know what do you want from it. d-jon will be free forever, and I will create some other non-free tools (billing, team servers, chat tools, mobile versions, etc) around it that will bring some food to my family :D.

I'm working on d-jon beta 1.2 right now and It will have some cool features like the ones you could see in the following screenshots:


new log view that will let you know what was the tasks you work on each day


new time by date view that will let you group and know the times spend to each task every day.

And some other features that I will post in some other blog later.

If you have any comment about the tool, or the article, please drop me a line with your thoughts, I will create a repository of "d-jon friends" and give away some versions of the new tools for free, also I will give a free android phone or iphone (I didn't decide that yet) to those of you that could say what do you like about d-jon and why do you think d-jon is useful.