[Programming] Social model

[tomkeus]

OK, it's time to say a word or two about social model employed
by the game.

In real-life cities, people move into towns, move out, they are
born and they die. While they're alive, they go to school,
work, spend money, have children, retire and do some other
stuff.

So, in order to get some social realism, simulator should be
able to encompass these things.

First, we have to take a look at some of the basic parameters
which will describe a person to the simulator. We could have
something like this

Code Select Expand


Person
{
    BirthDate;

    %Education level. This will probably%
    %be divided in a number of specializations%
    Education;

    Employer;

    Address;

    %This function will do various calculations%
    %of things such as satisfaction, decisions%
    %whether to look for new job, move out of%
    %the city and so on%
    Update();
}


If you look at the Economic Model thread you will see that
there is similar framework proposed for representations of
in-game businesses. This is done so there could be unified
underlying layer of classes exposed through Python so players
with modding inclinations will be able to use scripts to
inherit these classes and for example override Person's
Update() in order to implement different behavior of their
citizens.

Now, since cities grow, eventually, simulator will have to be
able to deal with millions of citizens. Brute force method of
trying to update every citizen's state isn't wisest choice. In
order to optimize this I suggest to use breakup of citizens
into different age groups.

So here they are: Children (0-6), ElementaryPupils (7-14),
HighPupils (15-18), Students (19-22), Workers (23-65), Retirees
(66-...). I think this is self-explanatory.

Why so? Well because, each year we have to update states of
only fraction of each group. For example, each year only one
eighth of all elementary school goers finishes school and so
update shall be concerned only with them. Similarly, for
working population, only one worker amongst 42 other gets
retired so instead of updating stats for entire working force,
it is only necessary to do that for around 2-3% of them. Here's
a little diagram.



Now, update I just described is concerned only with ageing
process and education of the population. Each one of these
groups will be susceptible to other updates. Certain number of
workers will be updated for changes in income, job,
procreation, moving, death and so on. So, simulator shall never
have to deal with entire population at once, only it's small
subsets.



Choice of subset of each group for updating will be partially
random, partially determined by other things. For example, if
some company went bankrupt, all of it's employees are good
candidates for their stats to be updated within next simulation
step.

Implementation-wise, this system should be really simple to
make. Entire population will be stored in a list container.
Since list is by construction age sorted, age group division
can be implemented with couple of iterators. Simple enough to
be tested soon!

[Nique]

Hi, I'ts kinda funny to see that every update() there will be picked a citizen to die. The older, the bigger the chance? Is it possible to add cause of death? (fire, hart attack etc) so we can overlook all those death's and see what needs to be improved to prevent such deaths?

This could be tested in a WINFORM application. I think this is something reachable for me. This could be written in C# right?

[WC_EEND]

Will residents also have an IQ? Like for instance that dumber sims aren't able to go to college and thus get worse jobs and earn less (and the other way around)

[tomkeus]

Will residents also have an IQ? Like for instance that dumber sims aren't able to go to college and thus get worse jobs and earn less (and the other way around)

I don't plan to implement that. I plan to keep things as simple and general as possible. When such big projects are in question, each added feature will spawn exponentially growing number of bugs in later stages of development.

I have mentioned in the first post that it should be relatively simple to script new behavior model and new parameters for citizens. So if players want for their citizens to have IQ, have preference for blue over red color, Pepsi over Coca-Cola and such things, they will be able to do that.

[tomkeus]

Hi, I'ts kinda funny to see that every update() there will be picked a citizen to die. The older, the bigger the chance? Is it possible to add cause of death? (fire, hart attack etc) so we can overlook all those death's and see what needs to be improved to prevent such deaths?

Probably. Some statistics database will be kept. Although, it is worth mentioning here, that due to the semi-random nature of update samples, player probably wont have precise and detailed statistics about city.

From my point of view that is good, because it is closer to the realism. Because governments and city councils don't always have precise and complete data on their population and economy. They rather have estimates. If they want good data, they have to perform census, market researches and so on. I think, that will be featured in the game. If player wants some good information he will have to perform research. It will cost money and take some time. During that time simulator will perform brute force query over all in-game entities and compile detailed statistics.

[Jonathan]

So will say less fire protection (stations) mean more deaths by fire than normal?
And the same with hospitals and heart attacks etc.

Jonathan

[tomkeus]

So will say less fire protection (stations) mean more deaths by fire than normal?
And the same with hospitals and heart attacks etc.

Jonathan

Yes, crime, fire station coverage, health coverage and some other stuff will be used to calculate death probability. Here's how that might look like:

-Simulator chooses citizen for update.
-Simulator calls citizen's Update() function.
-Update() function reads Address parameter.
-Based on that it queries Hospitals, Crime map, Pollution maps, Fire services and maybe something else.
-Death probability is calculated.
-Based on that probability citizen is either killed or left to live.

[tomkeus]

I would like to modify division I presented in first post. Difference is that I have divided workers into 3 categories (unqualified - elementary education, ages 15-65; qualified - high school education, ages 19-65; and high qualified -university/college education, ages 23-65).

So, for example, for batch of citizens that is coming out of elementary school, Update() function will calculate branching probability (high school/job) and make a decision. Decision will be based on wealth, state of the city's education system and maybe something else.

Here's graph:

[Nique]

What software are you using to make such graphs

[tomkeus]

What software are you using to make such graphs

OpenOffice Draw

[Nique]

Ok, good to know. A picture says a 1000 words. I was searching for such a program to explain some things  ok back to topic

[tomkeus]

I just ran some basic tests to get some idea about the performance of the social component:

I have implemented few of the most important features of the social model I described. The test was, insertion of 1 million inhabitants of various categories (children, workers, etc.) into the population, pool, and then calling the update() method for of each of them. update() method was provisional, it consisted of a bunch of operations carried out on the STL string class, just to keep CPU busy, as it will approximately be, when update() is fully implemented.

My machine is Pentium C2D 4400 @ 2Mhz with 2GB of DDR2. One of the cores was 100% busy performing some quantum monte carlo calculation. This simulated GUI running on one of the cores. Under this conditions, insertion of 1 million inhabitants took around 3 seconds, and update() for all of them was performed in around 2 seconds. Program used around 300MB of memory.

In realistic running conditions, update() shouldn't be called for more than 10% percent of the total population, so I can expect, that performing just the social model simulation might take something on the order of 1-2 seconds for really large cities.

[Genocidicbunny]

Instead of doing an update all at once, why not split up the updating into smaller, more frequent blocks? Instead of doing a yearly update on 10%, why not do a monthly on .834% of the population? Or, split it up even more. Update could have a few sub-methods that help it calculate. If you place each piece of data into a separate method, you can have update call each method only when needed. Say for instance, we have the death data. People dont wait until the end of the year to die, nor even the end of the month. They just die. We could do a daily update for this, but thats a little too small of a unit of time for us to do constantly. On the other hand, we could run the death simulation once every week. This will provide a more organic flow to the deaths.
A similar thing could be done for births, with a small added step. Each week, a certain amount of pregnancies are created. From there, every week, some of these pregnancies fail (miscarriages, etc.) Then, once the pregnancy reaches 34 weeks, the chance for birth is increased, until around 40 weeks when the child has a 100% chance of birth in the next weekly update.

Now, for education, we can do something less frequent. Since education is generally a semesterly system, we can call the update 3x a year, in january, in may and in september. This will provide a more realistic simulation of how the educational system works. We could additionally do a once-monthly update with a heavy weight on the three months I mentioned before. This will better simulate self-learning and non-standardized learning, such as vocational learning, distance classes and self-study. At the same time, it will allow a way for us to weight when the educational system produces the most graduates and promotions to higher levels of education. It also gives a certain reality to the simulation for job availability. After all, every may, the lower income job market is flooded with new graduates, and thus unemployment in generally higher in that month.

Pretty much what im proposing to do is to split the simulation up into more sensical units. We do not need to do education update every week, but death/birth/immigration/emigration makes more sense on a weekly system. Job-wise promotions also do not need to happen too often either.
The advantage of such a system is that each person can run on their own update cycles, keeping track of which updates they need to do individually. This will spread resource load over the course of the entire game year, instead of concentrating it in one spot.

[tomkeus]

Instead of doing an update all at once, why not split up the updating into smaller, more frequent blocks? Instead of doing a yearly update on 10%.

The code I wrote, chooses every month  10% of citizens randomly. To that number are added citizens from targeted sample. For example, if some company busted, all of it's employers are updated next month.

Instead of doing an update all at once, why not split up the updating into smaller, more frequent blocks? Instead of doing a yearly update on 10%, why not do a monthly on .834% of the population? Or, split it up even more...

I don't want to over complicate in the very beginning. It will significantly increase number of bugs to track down and I don't need that when I'm trying to build system that will, for starters, just work and keep itself in one piece.

Now, for education, we can do something less frequent...

Education is implemented within aging process. As I explained, Population is divided into 8 age groups. For example:

Every September, children, old enough, are enrolled into elementary school and pointer to the school they enrolled into is stored. Every June, elementary school students, old enough, are either sent to low qualifications work force, or to the high school. Based on the school they go, when they're updated, education level is calculated. Now, since 10% of these students are updated every month, for 96 months (8 years) only very small number will be missed by random update. For these students, education level will be calculated simply based on the school they were going when they finished it. It is not very precise, but since such students will be very, very small in number, error will be negligible.

Remember, we don't want to track every citizen to finest details. What we want is overall correct representation of most important large scale social processes.


This behavior should be fairly simply to make more detailed, in the way you described. This is how I done things:

Data on each citizen is stored separately from the relevant class that implements behavior. There are 8 classes, one for each age group. Every class has Update() method, which is called by the PopulationManager's Update() function. Citizens Update() method performs necessary calculations. Within Update(), OnUpdate() is called, which is virtual function and can be overridden to implement additional behavior. So, if we're later wanna go for detailed model, kinda like you suggested, we can, without disturbing what already exists.


Anyway, thank you for the great input, because I need ideas like this. I cannot design basic functionality if I'm not aware of the future functionality that should be supported by the basic framework.

[Genocidicbunny]

The code I wrote, chooses every month  10% of citizens randomly. To that number are added citizens from targeted sample. For example, if some company busted, all of it's employers are updated next month.

Ah, I did not see that bit. Yeah, that is what I was thinking, although again, id prefer something more granular. My main concern is that if you do an update only once a month, it will get bogged down at the start/end of every month, especially if a player has a lot of citizens in his/her city.

But as long as you make a good framework, we can most likely change things up for the better later on, once we have something workable.

[tomkeus]

But as long as you make a good framework, we can most likely change things up for the better later on, once we have something workable.

Changing two lines of code will accomplish finer time step. The reason I'm opting for larger time step is that this provides us with bigger number of subjects. This is done so parallel computation could be used more efficiently. But that is not written in stone. Once basic framework is in the place, we'll run tests and determine the optimal time step.

[Atomius]

"Instead of doing an update all at once, why not split up the updating into smaller, more frequent blocks?"

This is how i do things in Atocity actually. Instead of having monthly updates for finite state machines such as a zone tile (i don't yet have AI and population is not yet treated individually but would operate similarly to present FSMs like zone tiles/developments etc) they are updated based on an alarm event that is local, not global.

The controller obj has alarm events for things like the weather, day/night and other universal applications, but for localized updates a localized alarm is used, set off at regular intervals from the objects creation point in time. Thus the updates are evenly distributed throughout time and don't slow down gameplay for large cities as they did in older versions of Atocity and games like Simcity 4, which on my old pc would freeze for at least five seconds during these monthly updates.

[JoeST]

so you are simulating every citizen....radical

[Atomius]

Individual citizen simulating is something i've wanted to see since 4 had that semi integration with The Sims with 'MySim Mode'

[Djohaal]

Hmm, couldn't the calculations be simplified by doing a "statistical" simulation of the sims? I recon your simulation takes a 10% of sample to update. Couldn't it be optmized if it worked like a reverse census? Only one fourth of the updated population would be analyzed, and then the results for those 25% would be extended to the rest? This could add a kind of "precision" slider to the simulation to make it adapt the quality of the sim to the available resources...

On a side note I'm amazed how similar this stuff is to the methods we use in 3d rendering to simplify light calculation :p