Whatevermind's House of Wonders

[whatevermind]

I know, creative title, right? That's a better title 

The purpose of this thread is to showcase all my miscellaneous projects that generally don't fit into any neat category. BATs will still be over at my BAT thread, which isn't dead, just moving very slowly as I try to learn the software. Fortunately, there's plenty of other software I do know how to use just fine  and it can be used to research all sorts of random things and come up with all sorts of random new inventions. Anyway, introductions aside, onto the first topic...



The Fantastic World of Flora Families

This all started over here, with some discussion that stemmed from vortext's amazing new tree controller. If you haven't checked out what's happening over there yet, you should. The topic of Flora Families was brought up, and it seemed like one of those topics that needed further research. In case you don't know, Flora Families deal with the exemplar property kSC4FloraFamilyProperty, found in Flora exemplars.

Maxis uses Flora Families on its small offering of flora, and many of the older flora mods, such as those by c.p. tend to include the same Flora Families as used by Maxis. A third flora family is generated by PIM-X when you create a new flora exemplar, possibly as a typo in the programming. These three Flora Families, 0xE8F02107, 0x48F02107, and 0xE8F02106 seem to be the only ones out there, following a very cursory search through my plugins. 

Which is to say there could be others in use, but it's not a common thing by a long shot. Flora Families are known to be the cause of the City Hall Random Tree Bug - whereby when you plop the Maxis City Hall, you sometimes get a random and entirely out of place tree on the lot. This is because the City Hall lot contains flora, but instead of pointing to any particular flora IID, it points to one of the Maxis Flora Families. When c.p. created all of his flora and gave them the same family ID, it opened up that lot to a whole new world of trees that can appear on it. This was, until very recently, the only information on this site about Flora Families, and the advice was "don't use them". Good advice, but imprecise. Better advice is, "don't use that ID".

As you've probably gathered, one of the (and perhaps the only) uses for Flora Families are to allow random flora to appear on a lot. They behave in this regard the same as Prop or Model Families, and are even called by the Lot in the same way. In the Lot exemplar property LotConfigPropertyLotObject, one of the options for Lot Objects are flora, and when you use flora on a lot, the 13th rep of that property can be used to point to a particular flora IID, or a Flora Family ID. This is how the City Hall does it, by pointing to one of the Maxis Flora Family ID's.

Now, a note about those Flora Family ID's - they don't seem to correspond to anything that I can see. They are not the TID, GID, or IID of any file in the .dat's. This has two important implications in working with them. First, it doesn't seem possible to use Flora Families in tree controllers. Second, you can make up any ID you want, provided there isn't some limit on this in the .exe. Which means we'll probably want a Flora Family ID Registry before this gets too far. 

The reason I don't think Flora Families have any future in tree controllers boils down to the fact that tree controllers don't have any single master file that could point to a Flora Family. Instead, they are made up of a collection of flora exemplars that work together to define the tree controller. Flora Families are a property of flora exemplars, they exist because some flora exemplar lists them. As I mentioned above, they don't correspond to any file of their own. This is important because of how tree controllers work. When you go to plant trees using a tree brush, the game checks various conditions, and points towards the appropriate flora exemplar for where you are trying to plant a tree. The flora exemplar in turn uses its RKT property to point to a S3D file for that particular tree, and the result you see is that model (S3D etc) file on the spot where you clicked your tree brush. RKT properties require the full TGI of whatever they are pointing to. Flora Families are not files themselves, so there is no TGI the RKT could point to. There may be a way around this, but it looks like a dead end at the moment.

So, I researched their other potential use: Lots. Following on the idea that you could randomly make up any Flora Family ID (FFID) and supposedly use that ID on lots to point to an entire family of flora, letting the game randomly choose one to place on the lot, I put together a series of new flora exemplars to test this theory, and gave them all the same, new FFID. The models in this case were the topiaries from BSC MEGA Props RT Vol01, because I needed something tree like that would be different enough I could ensure random placement. Actually, any old model would do, but I figured might as well stick with something that's actually a plant. I then created a new lot, a simple ploppable 1x1 with a small building and a flora. I modded the lot flora to point it towards my new FFID that I was using on the topiaries, and (after some trial and error, of course) fired up the game.



Behold, Topiaryville! I'll just let that picture say it's thousand words. Here's another:



As you can see, the game picks a flora randomly from the Flora Family, seems to give it a random rotation, and everyone gets a nice leafy animal in their back yard.

So there you go, Flora Families are very real, and very useable. Now, I'd be interested to hear if anyone else has ever tried doing anything with these, and what the results were. Happy modding! 

[whatevermind]

Well, that last one was quite the crowdpleaser! 

Anyway, until I do some lotting or create some flora, that's pretty much the end of that topic, which brings us to my latest little research project:

Exploring the Flora Brush

This project actually came about following the discovery that some terrain mods do not include a flora brush. The flora brush is an exemplar that contains the settings for God mode tree planting. So, for example, if you're planting trees in God mode, that tool you're using is the flora brush. Anyway, c.p. always included it in his terrain mods, and I had become used to those modified brushes, so when I found some terrain mods without it, it seemed necessary to essentially recreate c.p.'s supersize brush as a standalone plugin. I already did this actually. But of course I couldn't just leave it at that, I had to explore all the parts of that brush and figure out what they do and if there's any way to optimize it. 

The flora brush actually has only a few properties that influence its behavior, and these are common to all of the brushes, such as the terrain brushes, animals, etc. Basically, the brushes consist of components, and can contain up to four components that are essentially layered over each other. All brushes must have at least one component, some brushes use two. Each component then has a series of properties that are defined for it that determine its behavior.

You can see a screen grab from Reader of the original exemplar below.



The properties that influence behavior are all those that start with Comp1..., in the case of the flora brush, there is only one component. The rest of the properties are all related to game appearance, menus, etc., which I won't be getting into.

The first property there, Comp1TextureID, is the one that so far has captured most of my attention. Basically, it points to the IID of a FSH file that defines the "texture" of the brush. That part there is about as simple as it gets when it comes to modding. What gets crazy is the FSH file itself.

In theory, you could point that property at any FSH file, and the game would find some way to translate it into a brush, and would plant trees in some pattern depending on how it interpreted the texture. The problem is creating or finding a texture that you want the brush to plant in, and getting the brush to actually plant in that pattern.

The default texture looks like this:



If you've ever watched the flora brush in action, especially at larger sizes, such as used in c.p.'s Italia or Meadowshire terrain mods, you'll see that if you click and hold it in a spot for a while, it plants a big central clump and a series of outlying smaller clumps, then begins to rotate, and those outlying clumps become bands, until the full radius of the brush is filled with trees. Look at that picture above, and picture it spiraling, that's the default flora brush.

But what if you wanted it to paint trees in a different pattern? You'd need a different texture. So, as a tribute to SC4D, I made this:



Also, the presence of letters makes it really easy to see if it's painting correctly. We'll deal with how I handled spiraling later, for now, focus on the texture.

Now, in theory, if you select the flora brush and click on one spot and hold it, you should see that same pattern repeated in trees:



Wait, that's not right! I see a squished S, D, an some wavy lines below them. Hmm...let's try tweaking some of those other propeties:



Eh, that's even worse...



Nope, not that either...



Ok, now I see C and 4, still squished into the top half of the brush, but still those wavy lines below them. However, the lines are bunched closer, more of a big clump below the letters now. Now, this was done just with clicking the flora brush. But as you probably know, you can also SHIFT+Click the brush, useful for painting bigger areas. Oddly, I haven't tried CTRL+Click to see what that does, but I almost never use that on flora. Also weird is that I was getting a different pattern depending on whether or not I just clicked or used SHIFT+Click. Now, something to notice about that last picture, although it's not terribly clear, is that that block of trees below the 4 has a pattern. It's easier to see in the SHIFT+Click, which in this case produces S and D and a block of trees below them.



What you're looking at there is the more or less solid block of trees that appeared below the D. The S and D are hard to see in this shot, but trust me, they're there, just very blurry (this was due to some messing with the texture I had been doing that had blurred it), and mostly cut off. Now, look closely at where this block begins, and read it left to right, top to bottom, like text. There is a solid stripe of trees, then the next "line" contains a string of tiny clusters of trees separated by empty spaces. Keep reading, line by line. Looks like a numerical sequence, doesn't it? An increasing numerical sequence, wrapping lines. Each few clumps of trees, the size of the clump increases. I had been thinking up to this point that this block of "junk" trees that were planting below the letters that were supposed to be there was just random fill, but this changed my mind. It also proved to be a breakthrough in cracking the brush textures.

Til next time...

[Swordmaster]

Oh, I don't think I had seen your topic! Good research into the flora family property. This might be useful sometime to me.

I also have a standalone flora brush mod, but never looked at the texture. Doh! Great discovery. However, after running a first test, I don't understand at all how it determines the outcome of the brush. It doesn't look one bit like it. Needs further testing.

EDIT: The texture itself seems to be different from other FSH files. GIMP, for one, can't open it.


Cheers
Willy

[Lowkee33]

I missed the topic too.

I have quite a few flora families in my own lottings.  As flora, they provide a pollution reduction so have a different value to a Lot.  Main problem is that PIMX doesn't see them, and MaxisLE shows the spinning diamond.  My solution is to make the lot with a member of the family so that I can tell what tree I'm planting, and then open up reader and switch the values over so the lot uses the family.

I had the same initial results as you with altering the brush.

But Lordy, how can you leave us like that? 

[Silur]

Yes, it's a very interesting theme  ...
Thank You, Whatevermind.

[Gobias]

Wow - awesome discovery regarding the distribution texture.  I haven't tried messing with it yet but looks quite interesting.  I've always been unhappy with the ugliness of the god mod tree distribution - I felt like it plopped a lot of trees at the center of the brush but too many stragglers popping up around the brush edges made it hard to create clean forest-edges without plopping trees in MM.  Now I can see why!

[vortext]

Interesting! I always wondered why the brush would spiral, now I know! 

[whatevermind]

Thanks guys!

I think flora families are pretty neat. It's certainly something to think about for anyone working on new flora or MMP's. Unfortunately, going back and changing all the existing flora into new families would be a monumental task, and could always cause problems with any lots that were using them, so I don't know if it would ever make for a useful public release. And, as Lowkee33 mentioned, they're something of a pain to use in lotting. I had to use pretty much the same method to get "Topiaryville" all lotted.



Well, as it happens swordmaster is on the right track there, as these FSH files are no ordinary FSH files. They follow the FSH spec well enough, they're just another flavor. I'll have to work out the details and update the spec on the Wiki at some point, but I think most of the information is already there, at least enough to decipher the files.

As I had been testing different variations on the texture, I had already been thinking that some of my weird results were due to the way the game was parsing the texture. However, it was seeing what looked like patterns in the trees that got me thinking about the file type, and led me to pop open the textures in Reader's hex viewer and take a closer look at them.

First thing I noticed was that the textures I had been creating, using Null45's PNGtoFSH tool, looked nothing like the original texture. Both were clearly FSH files, but the ones I was creating were far more "advanced", for lack of a better description, especially in the compression of the files.

This, remember, is what the original texture looks like:



When you open that up in Reader's hex viewer, you see this:



Definition: bitmap. (or bytemap, perhaps  )

At this point it became pretty obvious that if I was going to create new brush textures, it was going to have to be by way of hex editing. The compression options offered by the PNGtoFSH tool are just way to advanced for these files. So, that's what I did. It's tedious, sure, but once you figure out what goes where, it's doable, kind of like ASCII art, actually.

The remainder of the file actually consists of a giant block of "text" that simply counts up the character list from 0x00 to 0xFF:



Anyway, a little trial and error determined what part of the file contained the image itself, and so I recreated my SC4D texture thus:



Result:



Beautiful! 

So that's the basics of changing brush textures. A lot of the other brush textures are far bigger than the 32x32 of the flora brush, so viewing/editing them like this in Hex is a little harder, but they all seem to follow this same basic format, so I think you could apply this sort of change to any brush texture you wanted. You could also make brushes simply 32x32. There's no reason they have to be bigger that I can see other than creating more room for detail, because the brushes use a different method to scale the texture to the correct size.

Now, I didn't get such a pretty square picture without messing around with all those other properties, and I got some pretty interesting results along the way. The site being down yesterday actually gave me the chance to work out some of the last bugs, too. However, I need to get my screenshots in order first, so that'll have to wait for now.

[vortext]

Nice going!

So I guess a collection of different flora brushes is soon to be expected?! 

Seriously though, are gradients possible with this - apparently rather simple - fsh texture? And if so, I wonder what it'll look like. 

[whatevermind]

Thanks! I don't know about a collection of brushes  but something, soon perhaps. Personally, I mostly just cover the entire city in trees, and selectively replant later as required, so I'd be making one that is a full square, and big enough to plant an entire big city with one click. However, each brush is capable of 15 different sizes, and you have pretty good control over the sizes.

I was thinking the closer zooms could be sized to more detail oriented brushes, as tiny even as 1 square, though I think you can take it as tiny as a single line of trees. I'd have to test that though. So, for a square texture, do you guys have any sizes in particular you'd find useful? Or is there a texture more useful than a square that would be worth making?

As to gradients, it is possible to write them into the textures, but the flora brush doesn't recognize them. The terrain brushes use the full range of greyscale however.

In other news, I've got the file format about 90% worked out. There's a bit near the end of the file, between the palette code and the actual RGB strings that has me baffled, but the rest is pretty straightforward. I've even charted the header parts to help keep them straight:



Looks like the developers took the opportunity for some shameless self-advertising! 

[vortext]

Personally, I mostly just cover the entire city in trees, and selectively replant later as required, so I'd be making one that is a full square, and big enough to plant an entire big city with one click.

How dense do the forests get with this one click? I'm asking because since I reworked the brushes it takes quite some time to get at the proper density, so a bigger brush would be usefull. However, would it also be possible to have a 'stronger' brush - so to speak - which forces the individual items to cluster faster? That would be of even greater help. 

[Swordmaster]

So, that hex bitmap or whatever it is needs to be 32 columns wide to represent the texture, right? I figure you could get there with reader 1.4, but not the 0.93 version. Annoying. . .
And how many rows does this have? 31 or 32?

Anyway, I concur with Gobias that these outlying bushes aren't always desirable. So I made a perfectly round brush and got a perfectly round forest. Yay!




Cheers
Willy

[vortext]

Nice work Willy! It's quite annoying indeed when flora ends up in pastures. 

[whatevermind]

Density is not controlled by the brush, that falls back on the tree controller and the Cluster/FloraParameters properties of the flora exemplars. All the flora brush does is tell the game where to plant, the game then plants in that spot according to the tree controller. The flora brush in particular is essentially binary - it understands two texture values: 0x00 (don't plant trees here) and anything else (plant trees here). Likewise, a controller that can't find flora for the height/moisture criteria would still leave a blank spot where the brush said to plant trees.

Strength is another thing the flora brush doesn't seem to understand. There is a property Comp1Strength, which is widely used by the terrain brushes to alter strength, but I couldn't come up with values for that property that had any visible difference on the brush's behavior. Certainly nothing within the usual range of values had any impact that I could see. The only thing that seems to control how fast/strong the brush plants is the computer's own abilities. There is a small, but noticeable, lag when planting an entire big city versus an entire small city. So it's not instantaneous, it will take some time.

To quote c.p. on the topic:

"Since it is so large, it will plant trees slowly, especially if you use certain custom tree controllers. But if you find some things to hold down the shift key on your keyboard and the mouse button, you can walk away from the computer and do something more interesting for several minutes while the computer busily covers your city with dense, complex forests."

Now, by using a full square texture, and ditching the shift-key requirement, I'd cut a little time out of this, maybe. c.p. kept with the default brush texture/behavior on his terrain mods, he just made it bigger. I still haven't tried this on a custom tree controller, and I am a bit concerned for lag issues. But, it should still be faster than using the standard size brushes and having to wave them all over the place for a few minutes.

So, that hex bitmap or whatever it is needs to be 32 columns wide to represent the texture, right? I figure you could get there with reader 1.4, but not the 0.93 version. Annoying. . .
And how many rows does this have? 31 or 32?

The default size is 32x32, but it doesn't have to be. The brush uses the Comp1Width property to determine how big the brush is. The texture image size is really just to cram more/less detail into the texture. The terrain brushes are typically larger, usually 64x64. For a solid square, it could probably be anything - though there might be restrictions, such as power of 2, etc. The way the file format is setup there doesn't seem to be any obvious lower or upper bounds to the image dimensions. When viewed 32 columns wide, there are 32 rows as well. And yes, Reader 0.93 can't go that wide, but 1.4 does. Just remember that if you do use a different image size to update the header bits to match the new size.

Nice circle, by the way 

[Lowkee33]

Sweet!

The circle is a good brush, but there is an organic quality to the maxis brush that I like.

It's possible to add brushes to the menu (rather than just replace maxis), so I think a good first brush would the the solid square.  I envision something like when you are in zoom 5/6 (maybe 4), the brush places a 2x2 forest.  In this zoom, hitting control makes it 1x1 and shift makes it 4x4.  The ideal situation would have the zoom 0 make something that could cover an entire large map with trees in one click.  I'm not too concerned about the time it takes to plant all of the trees, but what I do find annoying is when I've clicked a whole bunch, and then come back later, and there are still gaps in the forest.

[noahclem]

Excellent work exploring these flora brushes    Very glad someone's taking on this project which goes so well with other things under development right now 

[vortext]

Ah I see, too bad the flora brush seems to be more limited than the terrain brush. That said, a big square brush sounds like a good plan. I'd like to give that a go for sure! 

Also, I noticed when treeing in zoom 6 or 5, items take longer to cluster than they do when applying the brush in zoom 2/3. Is this true or just my perception?

[Swordmaster]

Ah I see, too bad the flora brush seems to be more limited than the terrain brush. That said, a big square brush sounds like a good plan. I'd like to give that a go for sure! 

Also, I noticed when treeing in zoom 6 or 5, items take longer to cluster than they do when applying the brush in zoom 2/3. Is this true or just my perception?

My perception as well, although I always thought this had to do with the processing side of things. Takes longer to process all tree clustering in higher zooms.

Cheers
Willy

[vortext]

My perception as well, although I always thought this had to do with the processing side of things. Takes longer to process all tree clustering in higher zooms.

Hm yeah, good point. I could use a facepalm emoticon now. . . 

[epicblunder]

sonuvafish!  how have i missed this thread?!

This is some very interesting work.  I too would be interested in the square brush, and trying out a brush large enough to cover a whole city tile (despite the fact it would make my computer create smoke  )