Breaking Ground: Terrain and Waterworks

[Gobias]

Well let's call this the ribbon-cutting ceremony for Gobias Industries, my newly christened production factory making custom terrain, water and texture mods for the benefit of the SC4 community.  Which is basically just me.  Daniel.

Long-time lurker but I've never really posted.  I've been tinkering with modding the game for many years for my own enjoyment.  Having enjoyed the use of so much content produced by the community I now feel the need to try and give something back.  Hopefully it meets the quality standard.

Though I've created everything from demand mods and lighting mods, to street tree mods, my primary interest has always been developing terrain and exploring the games application of them.  Folks here like Lowkee, CP, Orange, and others have pioneered terrain modding and for all of their work I'm eternally grateful.  I'd never have learned much of anything beyond the swapping out of 128x128 textures if it hadn't been for their tremendous work.  Both reading their threads and dissecting their work w/Ilive has provided me with the knowledge necessary to feel I can produce something worthy for the community.  So to all, esp Lowkee as I sought his recent advice before kicking off this thread - a HUGE thanks.

So here's what I'm up to.  My initial objectives are water/terrain based and trying to build off of some of Lowkees stunning recent progress in the area to create a varied selection of mods.  I'm an avid hiker and I initially sought to recreate terrain in my three favorite places to hike.   So I'm attempting to produce:

- Berner Oberland Mod
- Highlands (or Icelandic) Mod
- Pacific Northwest Mod

Each of these will be dramatically different in palette from the Appalachian mod Lowkee's already released.  And by "comprehensive" I mean that I intend for the mod to include optional installable water, rock, beach, inshore, offshore, sunglare, water gradients, terrain properties, lighting scheme including night-water colors and adjusted coloring of sunrise/set, terrain INI tables, adjusted sun angles to match the locals, and of course terrain texturing.

Some reference shots for the desired look:

First, the Berner Oberland in Switzerland.  The region features vividly bright green grasses even at high altitudes, bright limestone-rock textures, glacier-fed lakes that are teal to bright blue in appearance.



The second, the Highlands mod which also resembles the Icelandic terrain (esp at low altitudes).  The lowlands in Scotland and the Icelandic grasses exhibit a glow that's very unique.  Both the water in the Scottish lochs and in lakes around Iceland both share a dark, mirror-like, almost black appearance.  And at higher altitudes the highlands rock formations take on a brownish-red appearance in season.  The area also features much deeper terrain shadows as a result of the low angle/azimuth of the sun which I will manipulate.



And the third will be the Pacific Northwest, probably the most "standard" look of the three when it comes to terrain (though flora is a much different matter) as it's more characteristic to the western Canadian mountains and closer in appearance to the American NWest landscape.


Now as for the progress of the three above mods I'm already well on my way and I'll detail the progress in the next few posts over the coming weeks.  The last step I'm working on is to replace several "temporary" textures that I've borrowed from the Appalachian/Pyrenean mods which I color-adjusted to fit the palette.  I initially considered requesting permission to use the adjusted textures but I'm thinking now I'd rather just make sure that everything included in my mod is 100% made-from-scratch.  I kind of feel guilty color-grading someone elses texture, so...  give me a little time on that front and I'll have more to show.  Until then, a brief preview of coming attraction (with several temp textures in place):

Highlands:


... So that's the destination.

Before I get there I'm probably going to take some time to explore water mods in depth.  There's much to do with water that I don't feel most modders take full advantage of.  More than just replacing the base textures.  In my next post I'll cover some details regarding the Sun Glare:  Adjusting it to fit the mod and how it interacts with the water surface gradient.  Lowkee explored this subject on his thread however I'm finding a few additional details on the subject that I think begs continued discussion... (testing on the glare below)


Oh and before I forget.  PM is the best way to contact me (not email).  Suggestions, comments, criticisms all welcome.  Looking forward to getting started and I hope that the end result is a collection of mods that are LEX eligible.

Cheers,
Daniel

[catty]

looking forward to seeing where you go with this   

[cmdp123789]

Nice... I really enjoyed reading this... took some minutes, but it is perfectly understandable... there many features that are not covered by modders.. but it is because of their experience that we have this much today... so I totally agree that there is a need to take it to the next level.. so go for it! 

[Lowkee33]

Glad to see you start a thread Daniel

One thing about changing the angle of the sun: sometimes this makes the shadows of models appear incorrectly.  You might start to see the bottom of the shadow beginning again.

Don't forget about the Weather Exemplar.    It would be interesting to see someone else have a go at this.

[Gobias]

Thanks all.  I should have an update to show this weekend.  Work weeks for me get pretty hectic so most of my updates will probably post on the weekends.

Lowkee:  Yes, I will definitely have a go at the weather once the basic mods are cleaned up.  And thanks for the tip on building shadows, I'll definitely keep that in mind while testing it.  In the past when I've played with sun angles I find it necessary to scale them back from real-world figures or the effect looks too distorted in-game.  Esp in locales like Iceland where angles in the low teens around autumn just looks awful in-game.  So it's definitely a balance between reality and playability.  I'll watch for the building shadows as I go.

[Gobias]

Well the little time I've gotten to spend this week on SC4 has been spent in the water-lab experimenting with the sun glare.  The results have ranged from hideously awful to some halfway decent results.  Let me start out by introducing the approach to the water mod.  If you've worked on water before some of this may be old news but for those looking for an introduction to the process I'll walk you through some of the tools:

A lot of water mods in circulation merely replace the main game water texture (09187300-4) and go no further.  Here's what we actually have at our disposal:
-The water texture: 09187300-4

-Two shore/beach textures (13,14 textures).  Some modders use this as beach texture, others as extension of water.  I use them to create unique combinations of shoreline color blends.  Here they are indicated by red/green below:


-Multiple Terrain Properties including:
waternightcoloradjustment:  to affect the RGB change of water values after night falls
waterdepthtocolorgradient:  controls the distribution of gradient color (1 pixel relates to meters of depth)
min and max waterdepthfordeepseabedtexture:  controls the distribution of the 13-14 textures

-The sun glare alpha:  69efb7da  (a reduced version of the MAXIS glare is pasted below.  This is what I've been using to retain but mute the glare effect for a about a year now).


-Gradient c9efb727:  controls the color of the "side" of the water table from the top down.  (Picture below tests the gradient effect w/the gradient changed to RED):


-Gradient 69efb7da:  controls the "side" color of the water table and, most importantly, the top pixel of the gradient affects the surface color of the water.  (Pictured below w/gradient as RED)


NOTE:  Lowkee explored the effects of both gradients in his threads.  An approach he (and also drunkapple's in his recent STEX release) put forth that if the water texture is a grayscale then the top pixel of the 6-gradient can be manipulated to create an endless spectrum of water-surface hues without having to change the color of the main water texture itself. 

My testing this week has centered on the concept that if one intends to keep the sun glare in place (you can remove it by simply making the glare alpha a flat-grey alpha) then SC4 is actually applying the top-pixel color of the 6-gradient at varying strengths and blending it with the underlying water texture according to the strength and distribution of the sun-glare alpha.  Therefore it may add another tool to the modders toolkit to retain the glare and use it for a blending effect.  When the glare is "blackest" the 6-gradient hue is strongest.  Where the glare alpha is "whitest" the true color of the BMP shines through at it's brightest.  See below:

Here's a Bavarian Glacial lake texture blended by the Maxis Sun Glare w/a RED 6-gradient top pixel...


If we switch the underlying water texture to a darker "Atlantic" blue then the blend w/the red gradient is more purple around the edges and you can see where the base texture of the water is darker blue in the center (where the glare is whitest):


Using a flat-grey sun-glare alpha eliminates the glare and allows purely the color of the top-pixel in the 6-gradient to show across the water:


So that brings us to the glare itself - a nasty bit of business as it's proven to manipulate successfully!  With so many variables acting in concert my attempts to experiment with it have had mixed results.  I'll show a few below ranging from really-distracting-and-obtrusive to more subtle and possibly useful effects:


Here's a selection of glares I tested...






...and a few results... (that last one, yikes!)





So it's definitely still a work in progress and I may still elect to take the safe-route and simply reduce the glare as I have been in the past.  But it's an interesting concept because we might not even think of it as a "sun glare" at all.  What it really is is an alpha that can distribute texture across the entire map, not just across a tile.  That makes it uniquely useful since the texture of the water 09187300-4 can only ever be tiled.  This allows us to create a map-wide gradual effect.

I've had the best success combining an underlying water texture with a top-pixel gradient color of either SLIGHTLY different color or mixing something like a blue water w/green gradient and applying a low-contrast glare.  In other words we can use the sun glare not as a glare at all but as a method of spreading a blended water color across the map for a pollution-effect or a deep sea v. shallow sea effect.  Similar to how I typically use the 13-14 shore textures as a green/brown mixture in the below images:





... At any rate, very much a work in progress.  If anyone has any thoughts on the glare-conundrum I'd love to hear them.  Right now I've gotten to the point where every test I attempt with it ends up looking too busy and I think I'll have to stick with a simple muted-glare and come back to this a little later down the road. 

Next up is to rework some of my older base water textures (pictured in most of the pics above) with something a little more HD.  I always use a 5-zoom texture approach w/water because I don't think one should see full-on wave detail when hovering at 20,000 feet on furthest zoom.  I'll update once I clean up the base-water textures with replacements.  Then we'll deal with night-color adjustments.  And another topic coming soon... sunrise/set and changing the hue of both (using the 0917660E PNG) to match the locale.

Cheers,
Daniel

[Gobias]

... Oh and BTW I should mention that the glare-alpha does not have to be a 128x128... like any terrain texture it can be larger to increase the detail if the detail is actually needed.  The game will always stretch the alpha to fit the may.  I used 1024's in my tests.

[Gobias]

Took a quick break to work on beach textures.  Not 100% complete (closest zoom still has a small bit of a seam I need to clean up and it still feels a little too intense - I might try and normalize the contrast between the light and dark shadows in the sand some more...  but here's a preview...

[vortext]

Interesting stuff going on here 

The sunglare texture looks to be particular useful for a comprehensive water mod. As I understand it's basically an alpha texture applied all across the surface, right? If so, maybe it could be used to create more realistic waves. I could actually see that work much like the *yikes* example, albeit more subtle.

The beach texture looks pretty decent already though it maybe somewhat out of place in mountainous terrain. Perhaps a rocky, gravel beach is more in order.

In any case, I'll be following this thread from now on, keep it up.

[Gobias]

The sunglare texture looks to be particular useful for a comprehensive water mod. As I understand it's basically an alpha texture applied all across the surface, right? If so, maybe it could be used to create more realistic waves. I could actually see that work much like the *yikes* example, albeit more subtle.

Exactly correct, and that was the idea behind the "yikes" and several other tests with the glare - to use it as a map-wide wave texture rather than a sun-glare at all.  The "yikes" glare basically employed a photoshop ripple effect to see if we could add texture to the surface waves and mix that with a lighting effect.  Just came off way too strong though.  But it's an idea that I'll keep toying with as I go.

The beach texture looks pretty decent already though it maybe somewhat out of place in mountainous terrain. Perhaps a rocky, gravel beach is more in order.

Yes, agreed it's a bit too sandy for mountain regions and I'll use something rockier when I get into more specifics with the three intended mods.  For right now I'm just experimenting with cleaning up the basic textures in my arsenal and making a few from scratch to replace the "borrowed" textures I've used from a few other mods in circulation.  The beach pictured above I made yesterday as a substitute for the popular "Sandy Beach Mod" that I've used since it's release - a tall order because that beach mod is superb.  Once I've got a good selection of 100%-Gobias textures ready to go I'll start going into detail about the three regions I'm focusing on and begin tailoring the mods for each.

Thanks for the reply! 

[jdenm8]

That yikes example reminds me of the water in a certain RPG from 2006... It's very Gooey.

Great to see some progress here though, I had no idea the game employed greyscales like this to do things like glare.

[Gobias]

Work continues up in the Alps of Switzerland.  I'm still in texture creation mode and moving back and forth between water, rock, beach and grass textures.  I almost have the full arsenal of textures completed.  A few quick shots of work-in-progress near the Schilthorn above Lauterbrunnen valley.

Don't mind the texture distribution or the color grading just yet... I still need to work more on the INI table to determine distribution and once the base BMPS are done I'll go through and finely match them to the Oberland terrain color-wise.  Also many many textures not on display here since I haven't worked them into the texture table yet.

But since I feel like posting images now that the "borrowed" textures are completely gone.  BMPS are now 100%-Gobias.

[whatevermind]

Wow.  This is amazing, can't wait to see where you go with this.

[Gobias]

Quick update.  Wanted to show a little progress on snow textures.  I've always thought there's room for better high-altitude texturing in the game and most people just use pure snow or rock in their game.  So I created some varied snow textures w/drifts for the region.

Since the Berner Oberland is a glacial region I thought why not try and blend a glacial texture into the game using the flatter portions of the terrain texture map?  Here's the result so far:

Glacial texture:


... only applied at a certain altitude level and only applied on level terrain (not the highest rows but worked into the 8th and 9th columns of the snow-textured rows).


... and a little more texture in the snowy hills:


... the tail end of the glacial flow blends into the rock and decends towards the grassy slopes of the Alps:


Anyway... I'll work a little more on this effect but it seems to work quite nicely and gives a little life to the high-altitude game textures.

Again, I'm aiming for a mega-update pretty soon to discuss my improved beach, water, rock textures tailored now for the Alps.  Also I'll show some of the testing I'm doing w/the texture INI table and the exemplars that control strength and fade-distance of terrain textures when blending.  Color-grading is underway to match my terrain w/ the region so perhaps some to show there.  Eventually lighting will come into play but not sure how far I'll get this weekend.

I'm off to the cinema w/spouse and friends...

Enjoy,
Daniel

[Gobias]

First a little testing with the terrain INI.  I'm making the mods initially as non-seasonal (or weather-neutral) and once they're in good enough shape I'll attempt the weather-modding (no promises on results!).  So basically the terrain texture distribution is primarly goverened by:

- the Terrain INI texture map:  32 rows and 16 colums.  Rows correspond to temperature, but for our purposes since I've not yet attempted to regulate temperature in the simulator (and I've heard this isn't really do-able but I'll give it a go later on) then we can think of the rows as corresponding closely to elevation.  Fairly straightforward and allows us to blend in a snow cap pretty easily.  Columns move left to right from driest to wettest and here's where it gets a little trickier.  Without having yet played with the moisture simulator (Lowkee has done good work here in the past) if moisture is static then the center columns of the table (8-9) end up getting applied at the flattest slope of terrain.  The furthest columns - dry and wet - apply stronger to sloping terrain.  But it's more of a bell-curve since a slope can be either wet or dry depending on how the game applies moisture.  Better explained graphically.

I set up 6 flat textures (yellow, green, red, etc) and slapped a number on them 60-65 to display in-game.  I set up the INI table so that there is no difference in temperature ROWS but each texture applies only to select group of consecutive columns.

Displayed in game:


This allows me to see where the game is applying a certain column of moisture.  The yellow texture identifies that the center columns (8-9) of the table are mean-moisture and that the game identifies each side of a mountain range as either wet slope (blue-purple, or wettest columns) or a dry slope (orange and pink or the driest columns).

I'm still tinkering with the INI table and matching to the region as best I can - here's the work thus far.  I'm still working at the snow-cap blend but I will point out that this region has less progressive blending when it comes to snow... it's a pretty hard cap due to storm-activity in the mountains and shorter-seasons.


Other factors determining texture distribution:
-Terrain Properties MaxHeight:  this controls over what expanse of altitude the INI rows will be applied.  Raising the amount will spread the rows over higher altitude.  So one easy way we've always used to adjust a snow-cap without having to work on the INI table itself is to change this figure.  Lowering the max height will bring the snow cap lower but will also compress the distribution of all the non-snowcap rows across the map.  I'm using a map of the Berner-Oberland region and will match my INI and cap to that map for accuracy.  But I'll include a "snow-cap lowering" option since the BO-region is pretty dramatic in it's elevations and most maps would not reach heights capable of showing snow cover.

-Each texture has a Terrain exemplar which contains properties of texture blending strength and the zoom curve (or where the texture will fall off with distance.  Lowkee explains his testing on this subject quite extensively on his board so I won't go into detail here except to say that I'm utilizing the same principals of correcting the texture spread curve as he discovered.

- The texture tiling factor in the Terrain Properties exemplar is another biggie and, again, Lowkee's thread gives the details of his extensive work with this property.  I'm employing a similar concept w/hi-res textures (all 1024's) with my mod.

In other areas (yeah, sorry I bounce around a lot - keeps my interest level up so as not to get frustrated with one area of the mod) here's a little work on beach and water textures:

At high zoom, less wave texture which I feel is more photo-real:


Once you get closer the wave detail becomes more apparent.  A few quick props thrown in for scale.  Also, this is not a wavy surface as this is glacial lake, not ocean.  Lake Thun and Brienz are pretty calm.  And YES they are that bright blue/teal:


The beach texture has been darkened and is less sandy.  There really is very little beach in these areas as the tree-line in most places comes right to shore so I reduced the beach width and beach altitude in the Terrain properties to show a short, shallow beach.

Also created a 14-shore texture to match the water color of the glacial streams flowing into the lakes in the area.  The mod will display stream color as such:


Also, before you say "Geez that grass is bright green!" yep the terrain in the region is also vividly bright.  Here's a shot I took last fall for reference:


That's it for now... more work on the INI table to match to the region and then I might tackle lighting next weekend.

[Kergelen]

Amazing terrain works. I love how you are developing Berner Oberland Mod. The glacial texture looks great.

Interlaken and surroundings is one of my favourite natural place I've ever been in Europe and I have some parts (Lauterbrunen, Murren,..) in my region map

[noahclem]

Really exciting work 

I'll be following this closely 

[Gobias]

Busy week - haven't had time to update.  But I have been working "behind the scenes" on the modding.  A few random bits and then perhaps later this weekend I'll have time to produce a more in-depth update.

First, based on Lowkees research regarding texture tiling I experimented with this property to make sure my textures matched seamlessly at the edge of my map (which they do):



I used a .09765xxx tiling factor whereas Lowkee uses half that for the Appalachian (.04x and change).  Personally I prefer the results of applying a little less "stretch" to my textures with a .097 rate but it really depends on how close of a zoom you create your textures with... otherwise details like rocks can look either way too large or much too small.  This is a good balance for me.

Another area I've been working on is detail textures, eg flowery meadows, crumbling stones, etc.  Wife and I are headed out to the park this morning where I (armed with camera and alterior-motive!) will try and shoot a few elements I'm still looking to turn into detail textures.  When I get those totally cleaned up I'll show some work there.  Since I was showing a little work on the glacier texture before here's another go at it:



It's still tough though to get the texture to apply just the way I want it to only on the flat terrain, and I think if I open this mod up seasonally instead of creating a single-season "stagnant" mod then the moisture simulator could really throw a wrench into that plan.  Already we get some unintended texture placement even if I create the INI as non-seasonal... so you get the glacier texture MOSTLY displaying on the valley surface but also  occasionally showing up on the side of a cliff:



Still worth going for?  Thoughts?

Anyway, it's a nice effect and I'll keep playing with it.

I've also started working with the moisture and temp properties and - wow - yeah it's going to take a lot of tinkering to get anything passable.  The concept all makes sense but like the glacial texture example above there can be pretty messy results no matter how you play with it.  The game's concept of moisture and temp distribution isn't quite well suited to making the mod seasonal but perhaps with more fine-tuning I can get something that gets close enough.  Much more to come later on with that topic.

And yet another area I've been exploring is lighting.  We'll deal with this on several fronts:  day/night color adjustments to water surface, the lighting-transition PNG which controls the hue and timing of daylight/nightlight, and the shadow and sun settings which affect everything from terrain/flora shadow length/intensity/angle.

For right now let's just cover nightlight.

PNG 0917660E controls the color/hue of lighting throughout the day and by manipulating it you can produce not only darker night color but even change the shading of sunrise/set and adjust the length of day.  My intention is only to have an option in the installer of using a slightly reduced sunset-intensity as I feel the red in the MAXIS sunset is a little too strong and tropical.  I could create both a MAXIS and DARKNIGHT version if desired.  To the lab!:

Here's the original MAXIS PNG.  Note that night-time is lighter blue and the length of day is quite long - from left to right the game will display these hues as global lighting colors as the clock progresses:


So first I tried changing the length of day.  Worked like a charm.  Sun rises at 2PM and sets promptly at 7PM.  If you set your lighting to permanent "DAY" in game then the color of noon (which is now night-color) gets displayed.  Interesting:


So I decided to push the limits a bit and change the whole darn thing to GREEN.  Sun rise and set are shaded green.  Daytime has a hint of green in it.  Nighttime - a dark green instead of MAXIS purply-blue.


And the result:
Day is a little minty:


Sunset:


And night:


Very curious.  Again, in the end my intention is really only to have the option of slightly reducing the "redness" of sunset in my mods which I feel needs to be muted a little in an Alpine setting.  Something like this, which is a minor alteration of Simfox's darknight:


The effects are subtle and add just the right reduction to the intensity of a red sunset that I was looking for.

Now when it comes to changing the nightcolor I personally like Simfox's Darknight as opposed to the MAXIS night.  On my PC I use a slightly altered version of the Simfox PNG with a little dash of extra-darkness in the deepest hours of night as compared to Simfox's mod.  But as I'm not a BAT'er I don't know how much my changing the color of night, even slightly, would affect their modelling?

So - BAT'er feel free to chime in - should I stay away from tinkering with nightcoloring, even if the adjustments are subtle?

[JoeST]

PNG 0917660E is quite an interesting one, it seems. does changes in the other axis have any effect on things?

[epicblunder]

I'm really digging the glacial effect.  Keep up the good work!