• Welcome to SC4 Devotion Forum Archives.

SC4Mapper 2013 Version development & suggestion

Started by cogeo, January 16, 2013, 12:15:48 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

cogeo

edit 02/25/2013 by wouanagaine ( hijacking the 1st post )
Github project started at :
https://github.com/wouanagaine/SC4Mapper-2013
People who want to contribute or fork the repo, please take notice that any executable version ( win, mac, linux, etc.. ) should be uploaded on LEX ( will be uploaded by LEX admin under your name if you don't have LEX upload rights )





Just saw this! So we have a new version of SC4Mapper! Excellent!

From the user's perspective, the most notable new feature (among others) is of course the Config Editor (not that the others aren't important, but features like the ability to export very large regions for example, are interesting to few users imo).

Don't know if I'm too late (and you have the time, of course), but I would like to post an idea about an enhancement here. There are two "issues" with the maps and regions currently uploaded (not SC4Mapper itself):
- Map sizes: maps tend to become larger and larger, and while many players really like this, to many others this is just annoying, as such maps are actually not playable. Regions like 60x70km large are very unlikely to ever get filled. They also pose a significant burden on system's performance: while the simulator(s) run on a city-by-city basis, some calculations are regional. Have you ever tried exiting (with saving) a city tile in a very large region? It takes a looooong time! And many players are not interested in all areas of a "Greater X-City" region, most are actually interested in the urban area and SOME of the suburbs. But given that maps are the way the are, they render the whole region, and have to landscape (or at least load-and-save) many city tiles that they actually do not want. I had requested a map for my own city (Thessaloniki) from dobdriver/drunkapple, but I posted detailed specs about what (imo) would be a "good" map. The region ended up 28x28km large, and I'm never going to fill it up, although much of the area is taken up by the sea. How about a 60x70km region then?
- Region break-up may not result in good, playable city tiles. While mapmakers do their best, you often get small islands divided in two, or you are left with a very narrow land strip (between the city border and water) where you can't lay, for example a highway and a rail line, and still have space for (local) roads and zones, or you are unable to build bridges. In such cases what players usually say is "IF the border was A LITTLE far off the coast, it would be MUCH better"! They can change the config, but editing the config works in much bigger steps. And mapmakers can't really foresee all these, they basically decide the area to be included in the map, and then make a simple config (usually all-large tiles).

To solve the two above issues, here is my idea (requires two extra operations):
- Be able to define a smaller config, ie only a portion of the region (.SC4M or 8/16-bit grayscale or RGB).
- Be able to drag the config overlay over the region (with the mouse, and possibly the arrow keys, for accuracy). This way it will be possible to define the city tiles, borders, islands, rivers etc, EXACTLY the way players would like.
Then the "Save Region" operation (pls consider a different name here) would create a playable SC4 region, including only the area occupied by the config overlay.

I think this would be highly appreciated by players. Not a "request" though (I'm not going to start any new region), just an idea. Don't know if you have the time and patience, but I think it would be great.

wouanagaine

#1
Hi Cogeo

Very nice idea you have, the first part can be almost there, as you can now 'erase' cities from the config editor. I should easily come with a way to allow the user to crop the map to the config

I will try to see how I can make user move the config overlay in a memory efficient way as I'm still fighting the big size map memory usage from nichter85' post


Edit :

In this case, would you allow a move in any direction or only  on the left ?



New Horizons Productions
Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dmscopio ♦ dedgren ♦ emilin ♦ Ennedi ♦ Heblem ♦ jplumbley
M4346 ♦ moganite ♦ Papab2000 ♦ Shadow Assassin ♦ Tarkus ♦ wouanagaine
Divide wouanagaine by zero and you will in fact get one...one bad-ass that is - Alek King of SC4

cogeo

Quote from: wouanagaine on January 16, 2013, 01:03:47 PM
Edit :
In this case, would you allow a move in any direction or only  on the left ?
Can't get your point exactly I'm afraid, as the answer would otherwise be so much obvious to me: moving the overlay in any direction other than left, wouldn't result in getting out of the region's bounds? The config should only be allowed to move only within the region imo, but I may be wrong.

Quote from: wouanagaine on January 16, 2013, 01:03:47 PM
I should easily come with a way to allow the user to crop the map to the config
While I don't know how you have organised your data in memory, I suppose you have allocated a (large) memory block (or maybe some file mapping or what), containing all the region's elevation data, not per city (at least while you are processing them). Assuming this, I can't see how a different-sized config would result in higher memory usage. But "cropping" the map might not be the best possible solution, because then the player would export a (smaller) SC4 region, check the region ingame, only to find that the config should have better been moved "a little to the left", for example, and then have to repeat the procedure (as the new -smaller- region now no longer contains the cropped-out data). The ideal solution imo would be a slight review of the .SC4M file definition, ie allowing the config to be of a different size (smaller) than the .SC4M size implies. But this would require a slight review of the format as well, unfortunately (as you also need to save the config.bmp size AND its start offset). Don't know if the format is extensible (tagged or so) so as to allow extensions like these and still have older-version programs, like SC4TF, able to read back these files, so I don't have any idea how much work this could be. However this capability may be very well left out, as SC4M is a tool for the player, while SC4TF for the mapmaker, and the procedure is one-way only, ie the .SC4M files saved by the Mapper need not necessarily be able to be read by the Terraformer, and this would be quite acceptable imo.

Quote from: wouanagaine on January 16, 2013, 01:03:47 PM
I will try to see how I can make user move the config overlay in a memory efficient way as I'm still fighting the big size map memory usage from nichter85' post
As for a memory efficient way to move the overlay around, I have some suggestions to make, although I don't know what the Python interface allows and what not (my experience is from WinAPI and MFC). I guess the bitmap you display in the application's window (the region's overview) is something you generate on the fly, and it's much smaller than the region's size, as it contains only sample data (properly adjusted and coloured for the preview). Then you also need to create another similar bitmap, containing the config overlay; this bitmap should be drawn using a XOR raster operator and you have to keep the position it was drawn in some variable; then, in response to the WM_MOUSEMOVE messages arriving, draw the bitmap again (on the last position it was drawn) using a XOR ROP again (this will revert the drawing surface to its original state), and draw it once more (to its new position). Alternatively (if you are drawing lines instead of blitting bitmaps, to display the config overlay) you can slightly modify your code to work in XOR mix mode (the WinAPI function that does this is SetROP2(), don't know about Python). That is, using a XOR ROP twice at the same position (ie inverting twice) restores the graphical surface, thus not requiring any additional memory.

I have a suggestion about a small (this one is indeed small) improvement (it's display-only). I took me a few minutes to figure out that the pic you posted showed a region 6x6 small cities large, and a config with a big city tile at the bottom-right corner and some extra small tiles to the top and the left; also that the blackened tiles are the "removed" cities (they could have also been the area "out of the config overlay"). Therefore I would suggest a modification here, as described below:
- The region area would be drawn the way it is, only without any gridlines.
- The config overlay would be again as is, ie a grid of thin black lines (only it should now be moveable).
- The city tiles should be changed, you could draw a rectangle with a thick frame, coloured according to the city size (red, green or blue) inside the black gridlines (without overlapping them).
- City tiles removed from the config but inside its boudning rectangle (ie those marked gray or white in the SC4 region's config.bmp) could be grayed (set R,G and B equal) or they can simply be left unchanged (the user will be easily distinguishing them, as they are inside the config overlay -the black gridlines- but not inside a coloured city rectangle). I could make a pic if you want, but I think it's quite clear.

And of course, these need not (and should not) be drawn and moved together with the config overlay (the black grid), because what the user might want when doing so, is having as much information available as possible (eg beaches, rivers, islands etc) for accurate placement, and those coloured or grayed overlays do hide enough information. So only the black grid should be visible when the config overlay is being dragged.

It think the above must be quite easy to make, as the most difficult part (calculation of the rectangle coordonates, conversions between the actual data and the screen coordinates and units, etc) is already done. For example, you have alrady made the calculation were to draw the black gridlines for city #27, then it's not hard to draw a red frame on a rectangle four (2+2) pixels smaller, using a 3-pixel wide pen.

SC4Mapper is a fabulous tools as is, but why not make those little interface improvements?

wouanagaine

Quote from: cogeo on January 17, 2013, 02:29:47 PM
SC4Mapper is a fabulous tools as is, but why not make those little interface improvements?
Because it is almost midnight now and I have a hard time reading all you wrote :)

Joke aside, I get a version with less memory usage, I can load an RGB region of config size 132x64
Still I'm not happy with the usage and will do more optimisation


Quote from: cogeo on January 17, 2013, 02:29:47 PM
Can't get your point exactly I'm afraid, as the answer would otherwise be so much obvious to me: moving the overlay in any direction other than left, wouldn't result in getting out of the region's bounds? The config should only be allowed to move only within the region imo, but I may be wrong.
In the case of island I wish sometimes I can move the overlay, and it will create water from the void, but that work only for islands
Anyway I think like you it is better to just limit to the region

Cogeo I will get back at your post tomorrow, it deserves more time than I have now.

For people having problem with their \MyDocuments folder, ( you can see the splash screen and then it crashes ), the fix is working, I will upload a new version when the memory usage will satisfy me

New Horizons Productions
Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dmscopio ♦ dedgren ♦ emilin ♦ Ennedi ♦ Heblem ♦ jplumbley
M4346 ♦ moganite ♦ Papab2000 ♦ Shadow Assassin ♦ Tarkus ♦ wouanagaine
Divide wouanagaine by zero and you will in fact get one...one bad-ass that is - Alek King of SC4

wouanagaine

Quote
While I don't know how you have organised your data in memory, I suppose you have allocated a (large) memory block (or maybe some file mapping or what), containing all the region's elevation data, not per city (at least while you are processing them). Assuming this, I can't see how a different-sized config would result in higher memory usage. But "cropping" the map might not be the best possible solution, because then the player would export a (smaller) SC4 region, check the region ingame, only to find that the config should have better been moved "a little to the left", for example, and then have to repeat the procedure (as the new -smaller- region now no longer contains the cropped-out data)The ideal solution imo would be a slight review of the .SC4M file definition, ie allowing the config to be of a different size (smaller) than the .SC4M size implies. But this would require a slight review of the format as well, unfortunately (as you also need to save the config.bmp size AND its start offset). Don't know if the format is extensible (tagged or so) so as to allow extensions like these and still have older-version programs, like SC4TF, able to read back these files, so I don't have any idea how much work this could be. However this capability may be very well left out, as SC4M is a tool for the player, while SC4TF for the mapmaker, and the procedure is one-way only, ie the .SC4M files saved by the Mapper need not necessarily be able to be read by the Terraformer, and this would be quite acceptable imo
SC4M file format is versionned, so I can add things in it
however as I lost latest SC4TF source code, I can't be sure I coded the reading of files correctly for forward compatibilty ( I'm pretty sure it can only read the current SC4M version )
It can be done ( adding the proposed offset and smaller config ) but I'm not sure of the benefit, I don't really buy the fact a user won't only discover in SC4 that he has not align the config properly

Quote
I have a suggestion about a small (this one is indeed small) improvement (it's display-only). I took me a few minutes to figure out that the pic you posted showed a region 6x6 small cities large, and a config with a big city tile at the bottom-right corner and some extra small tiles to the top and the left; also that the blackened tiles are the "removed" cities (they could have also been the area "out of the config overlay"). Therefore I would suggest a modification here, as described below:
- The region area would be drawn the way it is, only without any gridlines.
- The config overlay would be again as is, ie a grid of thin black lines (only it should now be moveable).
- The city tiles should be changed, you could draw a rectangle with a thick frame, coloured according to the city size (red, green or blue) inside the black gridlines (without overlapping them).
- City tiles removed from the config but inside its boudning rectangle (ie those marked gray or white in the SC4 region's config.bmp) could be grayed (set R,G and B equal) or they can simply be left unchanged (the user will be easily distinguishing them, as they are inside the config overlay -the black gridlines- but not inside a coloured city rectangle). I could make a pic if you want, but I think it's quite clear.
I like the gray overlay over removed cities idea a lot

As for moving, my idea is that user won't move the config, but will move the region instead ( at least in term of visual effect )

Well I will do some experiences this evening


New Horizons Productions
Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dmscopio ♦ dedgren ♦ emilin ♦ Ennedi ♦ Heblem ♦ jplumbley
M4346 ♦ moganite ♦ Papab2000 ♦ Shadow Assassin ♦ Tarkus ♦ wouanagaine
Divide wouanagaine by zero and you will in fact get one...one bad-ass that is - Alek King of SC4

dobdriver


A lovely update to mapper Wouanagaine,

I like two features you have included, the first is the ability to export the png directly from the sc4m file. I think from memory that was something I posted in my wish list. the second is the ability to turn off the city overlay grid. (Not so important)

I also like Cogeo's idea to be able to cut and shut large maps, Many people have requested some of my maps cut and shut because they only wish to use a small part of it, while it doesn't take long it would mean they can do it as they want.

Oh, some large maps eh?

Hawai'i is 132 x 148.
Bruce and Grey is 120x 136.
LA is 144 x 96.

These should keep you busy Wouanagaine ;<)



Cogeo, most of the time taken to save large maps is to re-generate the region image. The bigger the map, the larger the image, the longer the time.



Nichter, To generate the largest maps I produce you will need to patch mapper and terraformer with this:

4GB patcher

It will patch the executable and make a back up also and will run straight out of the zip file.
SC4 will render 8bit maps up to 39 tiles square but to render the map of Cornwall I had to patch the SC4 executable, it will now render 8bit maps up to 56 tiles square.

A good alternative to photoshop to deal with 16bit tiffs and pngs is this:

Photoline

It is a 64bit proggy and almost similar tools to photoshop, plus it handles 32bit images far better than photoshop. written by a German mob so there is a German manual. It ain't free but at 59 euros good value I think, plus a big online support base via their forums.


Keep up this grand work.

Cheers
dobs

wouanagaine

Thanks for Hawaii link and words

I can now report that I've been able to load it in my working version and it use only 800Mb, which means no more patching
I however don't know when I'll be able to upload a new version ( during next week at most )





New Horizons Productions
Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dmscopio ♦ dedgren ♦ emilin ♦ Ennedi ♦ Heblem ♦ jplumbley
M4346 ♦ moganite ♦ Papab2000 ♦ Shadow Assassin ♦ Tarkus ♦ wouanagaine
Divide wouanagaine by zero and you will in fact get one...one bad-ass that is - Alek King of SC4

wouanagaine

Yild/dobdriver => good catch, will fix

SimCity4RushHour => what for ? Won't you be more interested in the SC4M format ?


So some progress :

new city overlay look : ( I'm removing a small city in this screen )


I now can offset the overlay in the correct direction


New Horizons Productions
Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dmscopio ♦ dedgren ♦ emilin ♦ Ennedi ♦ Heblem ♦ jplumbley
M4346 ♦ moganite ♦ Papab2000 ♦ Shadow Assassin ♦ Tarkus ♦ wouanagaine
Divide wouanagaine by zero and you will in fact get one...one bad-ass that is - Alek King of SC4

SIM CITY 4 RUSH HOUR

#8
Quote from: wouanagaine on January 20, 2013, 01:57:38 PM
SimCity4RushHour => what for ? Won't you be more interested in the SC4M format ?
Pourquoi faire wouanagaine, comme je fait que tu parle fançais que c'est même ta langue aussi bien que la mienne, je vais te le dire direct moi aussi j'ai python 2.7.3 et je suis intéressé donc par python comme toi et j'ai pas seulement la version Windows 32 bits de python 2.7.3, mais aussi sa version Windows 64 bits x64 de python 2.7.3 donc pour plus de mémoire ce serai Génial, et encore plus si tu mettais le code source directement pour l'interpréteur de python, comme ça ceux qui le veulent comme moi par exemple pourrais travailler dans python directement bien sur tu peut toujours indiquer les dépendances de type import et autre moins évidents, que tu à mis si elles ne sont pas faciles à trouver.

wouanagaine

Quote from: SIM CITY 4 RUSH HOUR on January 21, 2013, 12:57:34 AM
Pourquoi faire wouanagaine, comme je fait que tu parle fançais que c'est même ta langue aussi bien que la mienne, je vais te le dire direct moi aussi j'ai python 2.7.3 et je suis intéressé donc par python comme toi et j'ai pas seulement la version Windows 32 bits de python 2.7.3, mais aussi sa version Windows 64 bits x64 de python 2.7.3 donc pour plus de mémoire ce serai Génial, et encore plus si tu mettais le code source directement pour l'interpréteur de python, comme ça ceux qui le veulent comme moi par exemple pourrais travailler dans python directement bien sur tu peut toujours indiquer les dépendances de type import et autre moins évidents, que tu à mis si elles ne sont pas faciles à trouver.
Je creerai peut etre un projet sur github quand je serai content du code (qualité, commentaire, et efficacité), comme ca tout le monde pourra participer

I may create a github project when I'll be satisfied with the code, so everyone could do stuff with it, I think of a one month frame


New Horizons Productions
Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dmscopio ♦ dedgren ♦ emilin ♦ Ennedi ♦ Heblem ♦ jplumbley
M4346 ♦ moganite ♦ Papab2000 ♦ Shadow Assassin ♦ Tarkus ♦ wouanagaine
Divide wouanagaine by zero and you will in fact get one...one bad-ass that is - Alek King of SC4

wouanagaine

A quick update
Everything seems to work so far
the Hawaii region is certainly the biggest you can do as map creators without any patch

So far:
loading as SC4M => 800Mb
loading as RGB => 900Mb
loading as PNG => 1.1gb
then exporting to SC4M, RGB or PNG can fail due to memory
At least I can read the RGB and export to SC4M as I think that is what most map creator is doing ?

Of course loading the SC4M and saving the region is ok, as it use almost 1.1gb

I've also made a modification for creating defaultconfig.bmp
So far it created a red config.bmp ie full of small cities. Now it creates an optimal config.bmp. So if you have a map that match a 15x15 small cities it will create a config.bmp for 4 big + 1 medium + 1 small in each direction

BTW : here is the new save overview picture with a config.bmp with some offset and some removed cities


New Horizons Productions
Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dmscopio ♦ dedgren ♦ emilin ♦ Ennedi ♦ Heblem ♦ jplumbley
M4346 ♦ moganite ♦ Papab2000 ♦ Shadow Assassin ♦ Tarkus ♦ wouanagaine
Divide wouanagaine by zero and you will in fact get one...one bad-ass that is - Alek King of SC4

JoeST

is there vertical offset too? that is such a cool addition, thanks Wou :D
Copperminds and Cuddleswarms

wouanagaine

Quote from: JoeST on January 21, 2013, 02:35:13 PM
is there vertical offset too? that is such a cool addition, thanks Wou :D
Yes you will be able to move on any direction provided you erased outside row/line of cities in that direction





New Horizons Productions
Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dmscopio ♦ dedgren ♦ emilin ♦ Ennedi ♦ Heblem ♦ jplumbley
M4346 ♦ moganite ♦ Papab2000 ♦ Shadow Assassin ♦ Tarkus ♦ wouanagaine
Divide wouanagaine by zero and you will in fact get one...one bad-ass that is - Alek King of SC4

wouanagaine

#13
A cool new feature ( IMO )

got a big region and want to play on a sub part of it ?

open the SC4M and go in edit config mode


with the mouse drag over the place you want to extract


the config will adapt at best


Of course as you have invalidated most of the cities you can move the overlay on any direction to get exactly what you want

You can directly save the region, or export the sub region

on importing this region, you got only the needed data





New Horizons Productions
Berethor ♦ beskhu3epnm ♦ blade2k5 ♦ dmscopio ♦ dedgren ♦ emilin ♦ Ennedi ♦ Heblem ♦ jplumbley
M4346 ♦ moganite ♦ Papab2000 ♦ Shadow Assassin ♦ Tarkus ♦ wouanagaine
Divide wouanagaine by zero and you will in fact get one...one bad-ass that is - Alek King of SC4

Swordmaster


MandelSoft

That would be excellent for those systems that can't handle big maps ;)
Lurk mode: ACTIVE

dobdriver



Oh it is guys!
Wouanagaine's got himself a boomer with this mapper.

It's very efficient compared to previous versions, It loads Hawai'i using only 230MB of ram and will load a map 52 tiles square equivalent to 212.992km per side. And that is unpatched.

Now SC4 can render a map 56 tiles square or 229.376km per side but SC4 needs to be patched to do so.


Cheers
dobs

Swordmaster

That's almost the size of Belgium. Meh  ()what()


Cheers
Willy

whatevermind

I love where this program is going. I've never much used mapper, being more inclined to fire up terraformer and make my own regions. But I see huge possibility in being able to play some of these monstrous regions that are out there, and pick the most desired bits and play them at full beautiful detail, without the loss that comes with scaling them down.

Swordmaster

Would anyone be interested in this color scheme?




Cheers
Willy