Menu

LEX File Exchange
EA Support Files
SC4 Wikipedia
Network Addon Mod
Dependencies
Chat

Author Topic: Using Blender (open source modeling program) for content creation.  (Read 19263 times)

0 Members and 1 Guest are viewing this topic.

Offline tomvsotis

Re: Using Blender (open source modeling program) for content creation.
« Reply #80 on: February 16, 2019, 06:34:06 AM »
Thanks Tom and Matt, I really appreciate your replies, and will take advantage from them  ;D

 Tom : given you've had more Blender experience that most of us some things that would be useful to know:

a) How do you record Macros and get them out of Blender into a script? Using macros tells us a lot about how to code the scripts later and may save us quite a bit of coding. Its a trick I use in Excel when I havent got a clue how to code something - recording actions as a macro and then editing it is easier than inventing it from scratch.

Hmmm. I don't know that you can *record* macros in the Word/Excel sense, but here's the next best thing: every command you enter in Blender is logged in the program's internal console. So you can effectively "record" the code for your actions by performing a few actions and then checking what the code for them looks like in the console.

The console is viewable by switching one of the viewports to "Info" mode. You can see it demonstrated here, from about three minutes in. Note that the technique he uses at about 3:25 - dragging the divider immediately above the main viewport downward to reveal the info viewport -- doesn't seem to work in Blender 2.80. Instead, the easiest way I've found is, using the default "Layout" workspace, drag the animation viewport (at the bottom of the screen) upwards:



...and then change it to "Info":



Then, as you go about your modeling, you can see the Python syntax for what you're doing. As an example, here's the log for selecting the default cube, scaling it 200% along the x axis, entering edit mode, and then applying an edge loop along the x axis - as recorded in the console:

bpy.ops.object.select_all(action='TOGGLE')

bpy.ops.transform.resize(value=(2, 1, 1), constraint_axis=(True, False, False), constraint_matrix=(1, 0, 0, 0, 1, 0, 0, 0, 1), constraint_orientation='GLOBAL', mirror=True, proportional='DISABLED', proportional_edit_falloff='SMOOTH', proportional_size=1)

bpy.ops.object.editmode_toggle()

bpy.ops.mesh.loopcut_slide(MESH_OT_loopcut={"number_cuts":1, "smoothness":0, "falloff":'INVERSE_SQUARE', "object_index":0, "edge_index":11, "mesh_select_mode_init":(True, False, False)}, TRANSFORM_OT_edge_slide={"value":-0, "single_side":False, "use_even":False, "flipped":False, "use_clamp":True, "mirror":True, "snap":False, "snap_target":'CLOSEST', "snap_point":(0, 0, 0), "snap_align":False, "snap_normal":(0, 0, 0), "correct_uv":True, "release_confirm":False, "use_accurate":False})


This code can then be entered into the program's internal text editor and saved/exported as a Python script. (Note that all such scripts need the initial command "Import bpy", which imports the Blender Python module.)

Hope that helps?
 
b) How do you make/automate LODs in Blender? - given in Max they're just Bounding Boxes, but we'd like better than that

Bounding boxes are easy enough, and manually making LODs/exporting them as .3ds files to gmax is also pretty straightforward and do-able. I haven't tried automating it, but I can't imagine it'd be that difficult to interrogate the program to get the bounding box of a scene and then draw an object to the scale of that box. I can certainly look into how to do this.

Offline rivit

Re: Using Blender (open source modeling program) for content creation.
« Reply #81 on: February 16, 2019, 06:23:22 PM »
Tom,
  thats terrific! It might be a bit clunky but it means that we don't actually have to imagine all of the code we need - just by manually doing the actions we need we can get a lot closer than we think. I'm sure that's going to get a lot of use and be a huge time saver.


Progress Report: Having taken a somewhat dangerous dive into the BAT4Max scripts (I didn't know how deep they'd be) it turns out that almost all of the vast code change that CP did was to handle the fact that the MAXIS executables no longer worked in 3dsMax. A couple of others fiddled with it (quite safely) and then Simfox added/changed things to deal with his TruNight and some other snafus 3dsmax threw up (over) itself.

All up this has made a significant (albeit working) mess of the BuildingMill - the Render routine is HUGE -  and I think sticking to gmax principles and finding out what Blender can and can't do is the way to go. However I will still plough through the Bat4MAx stuff to make sure I've seen it all and understand it as best I can.

Nuggets to come from this are the Sun,Sky and Exposure Settings Simfox defined for truNite variations
To be found here https://1drv.ms/b/s!AphvaLJG-tShg4wkcDAenWlkTsbRbA is a Pdf of a preliminary pretty print of the relevant details from those routines.

TODO: Art types for whom this means something are requested to see if/how Blender does these things and incorporate them in the code base. The Recording mechanism Tom outlined above will be quite handy.

TODO: There are some questions that need answering about Sun/Sky Colors and Sun Positioning and these answers might lead to changes in the Bat4Max scripts.

TODO: can someone take the Sun/Sky/Camera/Target objects in a Bat4Max and a gmax scene and dump/print out/record all of their attributes so we can compare these properly. We need to know they are absolutely the same/equivalent and it would be advantageous to record/set these in the scripts rather than have them float around in some scene where no one can see them.

TODO: can Null45 who I think made the FSHwrite.exe encoder tell us whether there were any fancy things in there or if it was all vanilla encoding. All the FSH made are DXT3 when the Day ones are DXT1 in SC4 so are 2x larger than necessary.  I can also do this with the code I have for GoFSH and can also now do VariableNight but would like to know if there are any "specials" in this code to worry about re truNite.

TODO: Who wrote/found the Datcmd.exe that makes the .SC4Model up? What are the command line parameters needed to run this?  (MGB may have answered this already).  Is source code available?

TODO: Still havent worked out how the S3Ds are encoded. Presumably I will find out once I get through BuildingMill.

TODO: What platforms are we intending to support Windows/Linux/Apple?  Current stuff is very Windows-centric - it affects .exes more than anything - in principle it could all be python (which would need to be written from scratch) but we have existing code in .net which could be ported (read largely recompiled) to .Net Core. Also there's quite a bit of java code about from earlier contributors. Basically we need to be able to do Bitmap manipulation (which is OS dependent), FSH encoding, S3D encoding, DAT assembly. Everything else seems inside the 3DTools.


  Sorry but TODOs will stack up for a bit - we are starting to get into the nitty-gritty of how things are made. This is a good thing.
   Ron

Offline vortext

  • Administrator
  • Forums Guru
  • *
  • Posts: 3361
  • Total likes: 569
  • Reputation: 23
  • CL:
    Street Smarts
Re: Using Blender (open source modeling program) for content creation.
« Reply #82 on: February 16, 2019, 06:54:29 PM »
Fitting a LOD was on my radar for a while already and as far as I can tell there isn't an straighforward way to get a bounding box which encompasses all the objects in scene. Few answers I found online boiled down to; get the bounding box for all objects and work it out from there. So that's what I did today, and actually just finished it.



Perhaps there's another way and it'll be proven obsolete but hey, at least I'm a little more familiar with the Blender api. In terms of code it's less than a 100 lines atm (here in case interested ), though I made some assumptions to speed things up.
Notably starting from the world origin to find the LOD dimensions. So say for example a scene contains a single offset object (e.g. the x dimensions for the bound box are -12,-5) the fitted LOD will extend towards origin anyway (-12,0) and thus the LOD width will not be correct. I figured it an acceptable compromise for the time being.
« Last Edit: February 16, 2019, 07:13:32 PM by vortext »

time flies like a bird
fruit flies like a banana

Offline rivit

Re: Using Blender (open source modeling program) for content creation.
« Reply #83 on: February 16, 2019, 09:22:08 PM »
Looks pretty functional to me. If you initialize the boundingbox with large negative maxes and positive mins then it will  encompass only the extent of the object by the time you've fed everything through it - even if it was only one point.

Is there no Blender function to get the Object Extent? == BoundingBox, or its max or min dimensions?

Offline tomvsotis

Re: Using Blender (open source modeling program) for content creation.
« Reply #84 on: February 17, 2019, 12:19:39 AM »
Tom,
  thats terrific! It might be a bit clunky but it means that we don't actually have to imagine all of the code we need - just by manually doing the actions we need we can get a lot closer than we think. I'm sure that's going to get a lot of use and be a huge time saver.

Yay! I'm glad it's helpful :)


TODO: What platforms are we intending to support Windows/Linux/Apple?  Current stuff is very Windows-centric - it affects .exes more than anything - in principle it could all be python (which would need to be written from scratch) but we have existing code in .net which could be ported (read largely recompiled) to .Net Core. Also there's quite a bit of java code about from earlier contributors. Basically we need to be able to do Bitmap manipulation (which is OS dependent), FSH encoding, S3D encoding, DAT assembly. Everything else seems inside the 3DTools.

Considering that Blender and Python are both cross-platform, it'd seem a shame not to support Linux and macOS, esp since they're already pretty poor relations as far as SC4 goes. The whole reason I started w Blender was that I only had a Mac, and thus couldn't run gmax/3ds Max. I think not spreading the ability to create BATs via Blender as widely as possible would be a mistake; we have an opportunity here to extend the BAT-ing community to people who might previously not have been able to join it. My 2c, anyway.

Offline vortext

  • Administrator
  • Forums Guru
  • *
  • Posts: 3361
  • Total likes: 569
  • Reputation: 23
  • CL:
    Street Smarts
Re: Using Blender (open source modeling program) for content creation.
« Reply #85 on: February 17, 2019, 04:59:45 AM »
If you initialize the boundingbox with large negative maxes and positive mins then it will  encompass only the extent of the object by the time you've fed everything through it - even if it was only one point.

Yes that would work as well, though imo the optimal solution would be introducing an additional step wherein a single bound box corner coordinate (in world space) is compared to 1) the other corners belonging to that bound box and 2) all other corners in the collection to figure out the min & max for x,y,z. That way there'd no need to initialize / make assumptions about the initial LOD size to begin with, as it would follow from said comparisons.

Is there no Blender function to get the Object Extent? == BoundingBox, or its max or min dimensions?

So yeah, I wanted to show you this but apparently there's no bound box display yet in 2.8 .. . Anyway, ample examples online of course, shown below the bound boxes as Blender fits them around objects.



Note the bound box location, dimensions and rotations are all in local space. As far as I've been able to tell there're no methods available to get the world space dimensions of a bound box direclty. Hence the LOD script first calculates the corners of a bound box in world space, and then iterates over all corners of all bound boxes.

Also Tom, are you able to export models as 3ds in version 2.8? If so, how exaclty?
« Last Edit: February 17, 2019, 05:15:45 AM by vortext »

time flies like a bird
fruit flies like a banana

Offline tomvsotis

Re: Using Blender (open source modeling program) for content creation.
« Reply #86 on: February 17, 2019, 05:37:24 AM »
Also Tom, are you able to export models as 3ds in version 2.8? If so, how exaclty?
Hmm, I hadn't tried in 2.8, and it seems there's no native export function. But if you want to export LODs to gmax, i think you're better off doing it as an .dae, as outlined by Fantozzi earlier in this thread:

Quote
If I export 3ds from Blender, most of the time it doesn't work in gmax for me ("improper format" or so).

You may try this:

a) download this converter written with gmax in mind (by a ms flight simulator fan): https://www.scenerydesign.org/modelconverterx/
b) from Blender export to *.dae format
c) import the dae into the model above converter and export as 3ds.

.....but I'm away from my Windows PC at the moment, so I can't check that it works :-/

Offline vortext

  • Administrator
  • Forums Guru
  • *
  • Posts: 3361
  • Total likes: 569
  • Reputation: 23
  • CL:
    Street Smarts
Re: Using Blender (open source modeling program) for content creation.
« Reply #87 on: February 17, 2019, 07:59:24 AM »
Hm yeah, shame there's no native s3d export (yet?), thought perhaps you had stumbled upon an alternative. I tried the .obj route previously (described here) which worked seemlingly okay but did require an additional import script for gmax as well. Guess automating LOD export should be put off for the time being untill the pros & cons of various methods are fleshed out.

Also regarding LOD fitting came to realize (while doing some cleaning  :D) a far simpler solution would be to just initialize the dimensions from the very first corner vertex in the list (instead of world origins, or super large values, let alone comparing all corners against one another), and proceed from there. And indeed that works fine, script has been updated to handle offset objects and added small check to keep from adding LODs to the scene.


« Last Edit: February 17, 2019, 12:44:30 PM by vortext »

time flies like a bird
fruit flies like a banana

Offline rivit

Re: Using Blender (open source modeling program) for content creation.
« Reply #88 on: February 17, 2019, 11:42:42 PM »
came to realize (while doing some cleaning  :D)
An excellent time to solve any sort of thinking problem... which reminds me I should probably do some.

I've continued to plough on through code (O so repetitive code - very little refactoring has been done) so don't have a lot to mention except that the exporting of the LODs seems a rather unnecessary evil - I read somewhere that its to 'not break the Maxis EULA' that we export to gmax and make up the s3ds from there. Why?

Call me a Noob %confuso but if all we are doing is importing/creating S3Ds from slices of a LOD mesh projection after mapping UV coordinates to textures and then putting both S3ds and FSH into a DAT, then why is does that break the EULA?  "$Deal"$ Reader can make S3Ds, and do packaging of DATs, just as well as a number of other tools manipulate dats, s3ds and FSH. Do they not break the EULA? Whats special here? We are not reverse engineering any Maxis executable code in this - we have reverse engineered the data.  I am reminded of the Unisys GIF patent...

Even more oddly, it seems the BAT4Max scripts still do all of the work that mapping and slicing entail but so far I can't find anywhere they are output, so its busy work that wasn't turned off - and a quite meaty part of the processing.

I'm also somewhat unimpressed with the Output file conventions/folders/names - leads to a lot of work internally in the scripts and is awkward to use externally. Maybe I'm just getting tired  &mmm

Also a pita about the no .3ds export - however code for writing them (very simple ones) does exist in NetWorld I expect, and is probably not that hard although messier than .objs which are very elementary. I have (written) code to read them .3ds and obj many years ago.  If I understand the process we really only need points and mesh triangles - everything else is deduced.

TODO: verify what image formats are output by Blender from a Render - in particular we would like 32bit ARGB bmps, but 24bit RGB will do but then we'll need two - one for color, one for alpha. PNGs are serviceable but much harder to manipulate natively in code - Windows does load them more-or-less correctly but doesn't save them well.

Offline tomvsotis

Re: Using Blender (open source modeling program) for content creation.
« Reply #89 on: February 18, 2019, 12:34:17 AM »

I've continued to plough on through code (O so repetitive code - very little refactoring has been done) so don't have a lot to mention except that the exporting of the LODs seems a rather unnecessary evil - I read somewhere that its to 'not break the Maxis EULA' that we export to gmax and make up the s3ds from there. Why?

Huh. I always wondered about this, too. It's particularly annoying in this context as it single-handedly undermines the whole cross-platform nature of the enterprise: SC4 is cross-platform (at least until Apple completely discontinues support for 32-bit apps, but that's a separate discussion), Blender is cross-platform, Python is cross-platform, but gmax... isn't. It's prob worth finding the EULA in question and seeing what it actually says.


Also a pita about the no .3ds export - however code for writing them (very simple ones) does exist in NetWorld I expect, and is probably not that hard although messier than .objs which are very elementary. I have (written) code to read them .3ds and obj many years ago.  If I understand the process we really only need points and mesh triangles - everything else is deduced.


Just for completeness' sake: there IS a .3ds export plug-in, but it's, well, a plug-in. It hasn't been updated for 2.80, but it did work (albeit not super-reliably) in 2.79. I have no idea how much work would be required to make it work in 2.80, but it does exist.


TODO: verify what image formats are output by Blender from a Render - in particular we would like 32bit ARGB bmps, but 24bit RGB will do but then we'll need two - one for color, one for alpha. PNGs are serviceable but much harder to manipulate natively in code - Windows does load them more-or-less correctly but doesn't save them well.

Blender's export settings for BMP are... pretty minimal:



...and, as per the manual, BMP export is 8-bit only. (That manual page is, afaik, the most up-to-date reference as to what formats Blender supports.)

PNGs are somewhat more flexible:



This is by no means my area of expertise -- i am good at photo editing, but not at manipulating image files via code -- but if we want lossless files w high colour depth, can we use TIFFs? (Or even TGAs?) Blender also supports 32-bit EXRs, but i know nothing at all about that format.

Offline tomvsotis

Re: Using Blender (open source modeling program) for content creation.
« Reply #90 on: February 18, 2019, 12:37:10 AM »
Or, wait, does "8-bit BMP" mean 8 bits per channel, i.e 24-bit colour but no 8-bit alpha channel? This stuff has always confused me  ()what()

Offline rivit

Re: Using Blender (open source modeling program) for content creation.
« Reply #91 on: February 18, 2019, 01:32:22 AM »
Yes, 8 bits per channel - so
the BMPs look to be BW = 1bit, RGB = 24bits  - BW for 1 bit alpha, RGB for color
and the PNG 1bit BW ?, 24bit RGB or 32bit RGBA
   
really wouldn't want to go TIFFs they can be huge, but TGAs are OK.
« Last Edit: February 18, 2019, 01:40:18 AM by rivit »

Offline mgb204

Re: Using Blender (open source modeling program) for content creation.
« Reply #92 on: February 18, 2019, 03:38:19 AM »
Call me a Noob %confuso but if all we are doing is importing/creating S3Ds from slices of a LOD mesh projection after mapping UV coordinates to textures and then putting both S3ds and FSH into a DAT, then why is does that break the EULA?  "$Deal"$ Reader can make S3Ds, and do packaging of DATs, just as well as a number of other tools manipulate dats, s3ds and FSH. Do they not break the EULA? Whats special here? We are not reverse engineering any Maxis executable code in this - we have reverse engineered the data.  I am reminded of the Unisys GIF patent...

I think the real issue is not so much whether we'd actually be in breach of the EULA, but whether EA would consider it a breach. In times past, EA was certainly keeping an eye on modding developments and it was probably just a case of being careful to avoid their wrath.

In the here and now, I'd imagine EA probably doesn't care and it's highly unlikely they'd notice such a thing, since I can't imagine SC4 modders are on their radar. But, should EA ever decide to send in their team of lawyers, right or wrong doesn't come into it, who could fight them? Simply in a financial sense, rather than factually or morally. That's ultimately the real problem, if EA decide they don't like it or think something is in breach, there wouldn't be an awful lot anyone here could do about it. It would have to be taken down from any sites or they'd sue us to oblivion very quickly.

Personally speaking, I agree, we're not re-using Maxis/EA code, so there is no actual breach.

Huh. I always wondered about this, too. It's particularly annoying in this context as it single-handedly undermines the whole cross-platform nature of the enterprise: SC4 is cross-platform (at least until Apple completely discontinues support for 32-bit apps, but that's a separate discussion), Blender is cross-platform, Python is cross-platform, but gmax... isn't. It's prob worth finding the EULA in question and seeing what it actually says.

Perhaps it won't make me popular, but SC4 is not multi-platform. Sure you CAN play it on Linux and a consistently borked Mac version exists. But few of the modding tools, including none of the official ones, ever worked outside of Windows.

The enterprise as I understand it, is not to make modelling possible on other platforms, although it would be a nice fringe benefit. The idea was simply to provide a foolproof method that would allow us to continue making quality models for use with SC4. The key issues being that 3DS Max and it's ever-changing nature, coupled with a lack of anyone to maintain the scripts for BAT4Max, have created a ticking time-bomb. Short on everyone returning to older versions of 3DS Max, which is very expensive and let's be honest, almost everyone is using the Student Licence to obtain 3DS Max for free. We'd be back to having to use gMax (SC4BAT) once more to create/render models, a huge step backwards. With Mental Ray being removed from 3DS Max recently, sans a re-write of the BAT4Max scripts, in a few years we're basically boned here.

I said when this was first discussed, one key benefit of Blender, as an open source application, we can always be sure a working version will be obtainable. Because we wouldn't suffer the forced-upgrade path that Autodesk are able to exert on everyone. So even if Blender outgrew the scripts for SC4, we wouldn't be deprived of a working solution as we could use an older version that did work.

I really feel that this should remain the #1 priority, ensuring a future-proof way of making quality models for SC4. If we can make it work on Mac/Linux, great, there is literally no downside to that. It could even transpire that this is something that works without expending any effort. But, if that is going to require any significant extra development time and resources, it should be very much a secondary or low priority feature. Why?, well simply because almost everyone making models already uses Windows. Just like almost everyone playing SC4 does. It makes no sense to divert resources to support a small percentage of potential users, in the first instance we should focus on what will work for the many, not the few.

Offline tomvsotis

Re: Using Blender (open source modeling program) for content creation.
« Reply #93 on: February 18, 2019, 04:57:01 AM »

Perhaps it won't make me popular, but SC4 is not multi-platform. Sure you CAN play it on Linux and a consistently borked Mac version exists. But few of the modding tools, including none of the official ones, ever worked outside of Windows.

True, although I guess the point I'm making is that one very appealing feature of the very nature of the solution we're looking at — i.e. Blender rendering + Python scripting — is that it IS cross-platform, and it'd be a shame to then tie it back to gmax if it transpired that doing so wasn't necessary. You're correct that the modding tools have never worked outside of Windows — believe me, I've tried 'til I was blue in the face to try to get PIM-X to work in Wine — but I wouldn't entirely discount the Mac and Linux community as far as playing the game goes. And anyway, BATing is a quite separate enterprise; as with any modding activity, there are some people who enjoy it more than the game itself. Ultimately, for the long-term health of the community, we should be trying to extend the capability to create content to SC4 as widely as possible.

Anyway, I think we're basically saying the same thing — that the priority should be getting an equivalent to Bat4Max working in Blender, and that if as a result cross-platform compatibility is maintained, that is, as you say, ultimately a fringe benefit. But I don't think cross-platform compatibility should be thrown out unnecessarily, either, so if we can find an alternative to tying the whole thing back to LOD rendering in gmax, we should.

Offline mattb325

  • BSC Team
  • Forums Parliamentarian
  • *
  • Posts: 1616
  • Total likes: 5271
  • Reputation: 25
  • CL:
    SC4D Housing Authority
Re: Using Blender (open source modeling program) for content creation.
« Reply #94 on: February 20, 2019, 02:16:49 AM »

The building is quite the things I had envisaged. My logic is such - if Blender can't actually match Bat4Max in outputs then we need to find out where, how to compensate and make a decision if its worth it. Same goes for handling imports/exports. To get there we need something to compare with. The gmax bit is so that we have target files with which to compare the outputs into SC4, as Bat4Max produces files that are 2x larger than gmax to start with.

This should also give the creator types something to get their teeth into as far as render settings etc. So thank you so far - brilliant, and I'm sure the Engineering Dept will find good use.


So I've finished the Engineering department. I've uploaded the game version on the LEX and will prep the model files so they can be dissected.



It has
* custom lod (just simple - the LOD is rotated 45 degrees, but it will be easy to troubleshoot rather than a really complex one)
* bitmap materials
* various 3ds procedural materials
* transparency + reflectivity
* basic + complex shapes (eg the foliage)
* standard primitives, extruded lines and meshes
* night lights + glare (i don't normally do glare)
* Day, MN and DN

So we should have most of the elements there to make a start.

I'll also take a further look at your todo items over the next few days to see if I can give any meaningful help there...
« Last Edit: February 20, 2019, 02:19:03 AM by mattb325 »

Offline rivit

Re: Using Blender (open source modeling program) for content creation.
« Reply #95 on: February 20, 2019, 06:57:53 AM »
Matt,

  that's absolutely a great test piece, totally fitting. It reminds me so very much of the Engineering School building I attended at Auckland University in the 70s. I suspect this will be a quite stiff test to meet as you've managed to pack quite a bit in there. Thank you very much, and challenge one for someone will be seeing how to get it into and then rendering it in both gmax (probably wont go) and Blender. I truly hope a few people will give it a go - even if just to try Blender - as this is real stuff to get on with.

My end, I've now transcribed the gmax scripts to VB.NET and have only 81 compiler errors remaining from the 8000 odd I started with 10 days ago. Even if it compiles that doesn't mean it works, of course, but I am nearly through phase 1 (unravelling how it does its thing) and I have started to sketch out the structure of a plugin - scaffolding classes and UI.

Still need to parse CP's BuildingMill for all the special Night Window stuff - which having never used, I don't understand at all from a process perspective. Found one or two coding errors in the gmax scripts along the way. Overall though the gmax code is reasonably tight within the context of scripting, and the prime cause of code 6 is probably unsuffficient or scrambled workspace/memory. I suspect its more likely if the LODs are trickier/large.

 Overall its been interesting, if tiring, to comprehend the 'spaghetti nature' of the meaning of the code caused by having a scripted language with 2 or 3 ways of referencing the same thing, everything implicitly typed and much global and declared on the run. Unravelling it and refactoring into clear classes without a lot of app dependencies is not going to be entirely trivial.

Research suggests we are going to end up exporting PNGs and OBJ files from Blender as support for BMP is thinner (maybe OK) and the OBJ format seems the most evolved of the export formats (although .3ds can be done). The Blend file format seems inpenetrable. Converting these to BMP and .3ds for LODs isn't that hard to do if necessary so we should be able to do the gmax trick at first. I would like to aim for removing gmax from the workflow in the longer run.

 In other news I read that Blender uses a XYZ axes with Y facing forward, Z up, 3dsmax is XYZ with Y facing backwards, Z up, and SC4 is XZY with Z forwards and Y Up. This affects thinking about transforms and other maths operations we use when comparing operations with 3dsmax. It also seems to have a fairly rich (if less evolved than 3dsmax) set of mesh operators. The renderer also appears to be reasonably advanced and can take advantage of Graphics cards for ray tracing. I will be very interested to see what its capable of.



Offline vortext

  • Administrator
  • Forums Guru
  • *
  • Posts: 3361
  • Total likes: 569
  • Reputation: 23
  • CL:
    Street Smarts
Re: Using Blender (open source modeling program) for content creation.
« Reply #96 on: February 22, 2019, 07:21:15 AM »
Sounds great Ron, if a bit tedious work but well worth it if eventually we can cut gmax out of the render equation.  :thumbsup:

In other news I read that Blender uses a XYZ axes with Y facing forward, Z up, 3dsmax is XYZ with Y facing backwards, Z up, and SC4 is XZY with Z forwards and Y Up. This affects thinking about transforms and other maths operations we use when comparing operations with 3dsmax.

Yeah this tripped me a up a bit when it came to making the camera object configurable using the game angles. However apart from a few minor quirks it does work as intended now (here) and won't be too difficult I reckon to hook it into gui later on.

Also found this method which, given a list of coordinates, will return the orthographic scale to fit all the coordinates in the camera view. It works as advertised, and takes into account camera nicely (i.e. returns different scale depending on rotation & zoom). However, it does not return offsets so it remains to be seen what use, if any, it will have in practice.


TODO: can someone take the Sun/Sky/Camera/Target objects in a Bat4Max and a gmax scene and dump/print out/record all of their attributes so we can compare these properly. We need to know they are absolutely the same/equivalent and it would be advantageous to record/set these in the scripts rather than have them float around in some scene where no one can see them.

Yeah I'm interested in this as well, would be much appreciated it we can get more info on the cameras, world settings and all related 'global' properties.

~~~~

The issue of wrangling the camera offsets has become managable by this super helpful method hiding in sidelines. Not quite sure if the orthographic scale is applied correclty (actually pretty sure it's not) but at least it provides a way to get the LOD in the upper left corner of the camera view consistently with a fixed number of pixels between it and the edges (janky code here)

« Last Edit: February 22, 2019, 02:18:42 PM by vortext »

time flies like a bird
fruit flies like a banana

Offline c.p.

  • BSC Team
  • Forums Senator
  • *
  • Posts: 645
  • Total likes: 880
  • Reputation: 12
  • CL:
    SC4D Green Thumb
Re: Using Blender (open source modeling program) for content creation.
« Reply #97 on: February 22, 2019, 11:37:48 AM »
I've continued to plough on through code (O so repetitive code - very little refactoring has been done) so don't have a lot to mention except that the exporting of the LODs seems a rather unnecessary evil - I read somewhere that its to 'not break the Maxis EULA' that we export to gmax and make up the s3ds from there. Why?

I did the original bat4max scripts, and I can assure you that EULA considerations had nothing to do with it, at least not for the original version of bat4max.  I had no programming background (Refactoring? what's that?  :D ), so was restricted to what I could do via maxscript.


Even more oddly, it seems the BAT4Max scripts still do all of the work that mapping and slicing entail but so far I can't find anywhere they are output, so its busy work that wasn't turned off - and a quite meaty part of the processing.

If I remember correctly, the LOD geometry is used in 3dsmax to calculate the size and position of the rendering.  (The slicing probably isn't used though.)


Still need to parse CP's BuildingMill for all the special Night Window stuff - which having never used, I don't understand at all from a process perspective.

If I can help with specific questions about the night windows code, ask away.  I'm not sure anyone but me has used the "night library" feature, so that might not be worth devoting time to.  The general point of the pixel-by-pixel post-render processing was to create an alpha channel for the night render, so only the nightlighted areas show up in-game.  (Which incidently makes them compatible with both "Maxis Night" and customs night mods).  That is also why a third render was done for certain types of nightlighting.  If you need to know what the specific gbuffer channel info is that the code is pulling from the render, I can figure that out for you, but a lot of that might be unneccessary/irrelevant in Blender.
« Last Edit: February 22, 2019, 11:43:44 AM by c.p. »

Offline mattb325

  • BSC Team
  • Forums Parliamentarian
  • *
  • Posts: 1616
  • Total likes: 5271
  • Reputation: 25
  • CL:
    SC4D Housing Authority
Re: Using Blender (open source modeling program) for content creation.
« Reply #98 on: February 22, 2019, 03:21:51 PM »
Awesome to see you here, CP...your 3ds mastery will be invaluable...way better than my non-existent script knowledge, that's for sure.  :thumbsup:



TODO: can someone take the Sun/Sky/Camera/Target objects in a Bat4Max and a gmax scene and dump/print out/record all of their attributes so we can compare these properly. We need to know they are absolutely the same/equivalent and it would be advantageous to record/set these in the scripts rather than have them float around in some scene where no one can see them.

Yeah I'm interested in this as well, would be much appreciated it we can get more info on the cameras, world settings and all related 'global' properties.

~~~~



Okay, so here's what I have so far: anyone please feel free to correct me.

In both gmax and 3dsmax "Zpx = 4000" is a constant between both programs in the camerarig scripts etc. In the gmax scriptutils, this is listed:

--------------------------------------------------------------
-- Get a ray corresponding to a pixel on the screen
-- only works with ortho cameras
-- Obsolete
--------------------------------------------------------------
fn FnSafeScreenToRay pixelPt = (
   local p0, p1
   local projXform
   local resultRay = stRay3D()
   
   --gw.setTransform (matrix3 1) -- paranoia
   
   projXform = inverse (viewport.getTM()) -- xform takes view space to world
   
   -- get point behind pixel in viewspace
   p0 = mapScreenToView pixelPt 4000.0 -- 4000 was empirically determined to be what mapScreenToWorldRay uses
   -- transform point into world
   resultRay.pos = FnTransformPt p0 projXform
   
   -- get point a little further behind pixel in viewspace
   p1 = mapScreenToView pixelPt 3990.0
   -- transform point into world
   p1 = FnTransformPt p1 projXform
   
   resultRay.dir = p1 - resultRay.pos
   
   normalize resultRay.dir
   
   return resultRay
)



That may be useful to you, I don't know....

As for camera/lighting here's some visuals on what I have taken from the systems. Note down the bottom of each pic is the XYZ co-ordinates of each element:

A) GMAX. I'm really rusty with the commands, so there could well be more to extract, but it will take some time for me to remember all of the commands (assuming they're needed, the lighting rig in gmax is very old....)


Lighting rig. Each individual light (8) has the same co-ordinates, which I find odd...I'm probably missing something


Camera Array

B) 3dsMax:






IES Sky, IES Sky Day Object Properties, IES Sky Target Object Properties

******************




MR Sun & MR Sun Object Properties

******************


TB2 Camera Handle properties


Hopefully this assists, let me know if anything else needed.

« Last Edit: February 22, 2019, 03:36:31 PM by mattb325 »

Offline rivit

Re: Using Blender (open source modeling program) for content creation.
« Reply #99 on: February 22, 2019, 06:26:03 PM »
@CP So very pleased to see you in this thread, you're like a missing master for me - I see your name everywhere on things I like and have in SC4.

I'm very impressed with the stuff you added into the building mill - given the awful environment to work with and the quirky script language I'm amazed that anything that big was built and worked. I had "sort of deduced" the alpha fiddling was to do with producing a variable alpha to fit with Maxis mode, but the depth of code still has me as a rabbit in the headlights. Must have taken ages to get it to work as intended. In fact, because I couldn't follow that code properly it made me find another way to make alphas for variable nights. I will ask if I have questions thankyou.
 
@Matt the stuff associated with the CameraHandle seems to stop us seeing info for each individual camera - I can't figure out a way to break that group into individual items. While we do have camera details (I can't remember how I found those back in 2009, so they need to be verified) we don't know the camera target in particular.

@vortext - glad to see you've found those little improvements - bit by bit getting closer to getting the views right.  The 'orthographic scale to fit all the coordinates in the camera view' is essential to framing the views and determining the end UV coordinates per sub mesh so that will be rather handy. In the BAT scripts this is then modularised into multiples of 256 for zoom 5, and smallest power of 2 to fit for the other zooms for the rendertexture sizes.

I have now got a compiled version in VB.NET.  It doesn't do anything useful but this has allowed me to identify/abstract those functions coming from the "rendlx" plugin, and from gmax/3dsmax in general. A pretty printed pdf version is available here. https://1drv.ms/f/s!AphvaLJG-tShg4wbSqa1_8cIZ4sy2w. It's also let me find those functions in the code that are dead i.e. not used. About 500 lines. I have started the design process on a new plugin structure - main goal will be to keep Blender code in one place so that the dependencies are easier to manage - we'll see how that works out...  :-\

  ASIDE: Refactoring is a process by which repetitive/verbose/complex code is abstracted into simpler functions to improve code quality by reducing the number of places mistakes could occur.  These new chunks can then be used everywhere - eg. if I need a single file name everywhere instead of writing it out in code each time in full I might refactor that into a constant which I then use everywhere. That way I can mistype it only once.