[wouanagaine]

Hi folks

A new version of SC4DP has been uploaded to a LEX near you http://www.sc4devotion.com/csxlex/lex_filedesc.php?lotGET=26


What's new

• This version is up to date with the latest finding in SC4 Loading order, faster, and provide some better compression which turns to speed up your loading time even more.
  
• You can now choose the folders you want to pack by selecting them in the mainscreen
  Folder bigger than 392 Mb will be automatically split by SC4Datpacker, and will produce more than one mega pack, so no more need for manual folders splitting
  
• A new dependancy is needed, so make sure you read the readme.html to get the link

Some results

• 1.18 Gb of plugins packed in 2 min, loading time to region 35 sec, loading time to a large developped city 1min20
• 3.46 Gb of plugins packed in 5 min, loading time to region 55 sec, loading time to a blank city 1min15
• Rough estimation for a 3.6 Gb of plugins, without SC4Datpacker, 5 min loading time, with SC4Datpacker 2 min

This version has been tested intensively ( as you may notice given the letter of the build in the zip ) but of course I can't for sure tell it is 100% bug free, so please if you have some problem running it ( and after you check the readme for dependancies ) please post your problem here and if applicable post as well as the file that SC4Datpacker can't pack


Take care and enjoy fast loading time

[JoeST]

ACE!!! another version. *installs windows just to test it out* 

Joe

[toxicpiano]

is this on the lex disk or do I have to download this?

[Andreas]

It is available at the LEX (the old download has been updated).

[toxicpiano]

It is available at the LEX (the old download has been updated).

so the lex disc version is outdated i take it?

[cogeo]

So there is a new version! Great!

Any thought of implementing the feature I had proposed earlier? I mean making it possible to exclude some folders from DatPacking by putting a marker file in the folder, eg DO_NOT_DATPACK_ME.txt (no contents)? Yes, the user can specify which folder to DatPack and which not, but it would be good to let the developer decide this too, not just the user. I'm talking about RTMT, which has optional plugins, and these may be changed during the lifetime of the RTMT installation. I think it would be impractical to expect the players to remember which plugins should not be DatPacked.

Please consider implementing this feature in the next release or update (I think it's quite reasonable to reckon that this isn't much work).

[wouanagaine]

Cogeo, I think you'll have a hard time explaining me why you don't want a user to pack a folder

I've just played with a 3.6gb all compressed, but the blgprop*.dat, even my working files are compressed, as well as your public RTMT version, without any problems.

If a player doesn't pack all its stuff, then he will have to think about where/what is loaded first, if it is needed to load stuff A in folder X before stuff B in folder Y which was packed in compressed plugins etc...
whereas using everything in the packed form, he just has to push the button and run the game, because if it works with all unpacked files, it will works with all packed files. That doesn't mean that if it works with all unpacked files, it will works with some packed and some unpacked files.

If anything change, like I useally change the pathfinding plugin in NAM depending of the kind of tile I'm playing, I just move in/out the correct file, and just run SC4DP and hit the 'start button. SC4DP only rebuild then NAM folder, that's it, nothing more, nothing to select or care about .

[Pat]

Great news for 2008 and also a new update for the anniversary of SC4D.... (Moved original post to the proper place....)

[High5Tower]

Here is my report about the new version...Its great!
My specs:
Plugins unpacked      3.80GB
Packed                   3.62
Maxis plugins (NAM)    .91.6MB
z folder & root files     .54.6 
Total plugins used     5.082GB
Comuter used: Dell 8400  Pentium 4 3GHz  2GB RAM
All files in both plugins folders are either datapacked or dat files, no model,lot,desc. Except for one
On first run I had 37.7MB of new stuff to pack in my various 21 folders that make up my original plugins.
It took around 5 minutes to do this and gave me a report with about 30 items I needed to remove and did so.
Ran again in about 1.5 minutes.
Settings used : Force repacking...NO: Optimize...Yes:  Include TGI...Yes. Datapacker reported 2.68GB
My average load up time has dropped by 1 minute.
I played for around 2 hours in many cities (18sec between cities) everything seems to work will.
Thank you for your great contribution to this game.

[bob56]

This is probably a stupid question, but what is the DatPacker? What does it do?

[High5Tower]

To put in simply it converts your basic building file that is usually made up of  a mode lfile, desc file, lot file into a dat file. And computers can read dat files a lot faster. As proof my game runs better now with 5GB of data packed plugins than it did when I only had 550MB of unpacked plugins. Hopefully there will be someone along that can give you a really technical answer.

[wouanagaine]

Good non technical answer, High5Tower 

Technical answer

Every file that SC4 is reading is made up of 'entries' ( like afile in a zip file ). The job of DatPacker is just to read all your SC4 files, extract all those entries and group them in less files ( originaly one per subfolder in your plugins folder ). So instead of having 10000 files in 25 folders, you'll now have only 25 files.

As each entry is uniquely indentified, DatPacker takes care of getting the correct one when you have duplicates ( this can happen when a file is updated, and you didn't read the readme of that file asking you to remove older stuff for example, or when 2 mods modified the same entry ) SC4 only use one. So instead of SC4 making the correct choice at loading time, it is left to DatPacker, this will of course make a win when loading SC4, because SC4 will only read an entry once

In OS low level world, opening 10000 files is a very long operation, so now SC4 will only open 25 files, this is a big win
Chances are high that your 10000 files will be scattered all over your HD, whereas 25 big files are more likely to be in the same place ( especially if you run a Defragmenter ) so your HD will locate and read them much more quickly

Moreover, it has been proved by users that the zoom bug CTD ( ie zooming in/out very rapidly can leads to a crash to desktop ) is related to the number of files SC4 is trying to read to get the correct graphics, so with 25 files you don't have this bug anymore

[bob56]

thats amazing! I'm getting that right now   

[shardis]

Hi, how can download this program? the link on top opens a window without possibility to download (1 photo, many comment, nothing more :S )
thx

[Swamper77]

You need to be registered to the BSC LEX before you can download. It is a separate registration from the forums.

-Swamper

[cogeo]

Wouanagaine, why is it so hard to believe that some folders should better be not datpacked?

In many cases the configuration of a pack is determined by the presence or not of certain plugins. And if the configuration can change during the lifetime of the installation, datpacking a folder (and having a single dat file) has two obvious disadvantages: a) the player is not able to tell if a certain plugin is installed or not, and b) removing a plugin that is merged into a common datfile is not possible (the player should delete the plugin and install it anew). In such cases having the option to skip a folder (and its subfolders) would be preferable.

And if you wonder if such plugins exist, NAM is the first that comes to mind. Surely, a DatPacked NAM will work, but then it's impossible to tell wheteher a plugin is installed or not, and it's also impossible to remove, say, turning lanes for roads. Therefore, to my opinion NAM should not be datpacked.

Many of my plugins can also be configured dynamically:
- RTMT props can be installed/uninstalled at will at any time allowing the props to be selected for each individual plop (eg design #1 along the main avenue, #2 at the suburbs and #3 in the industrial area); this is impossible if RTMT is datpacked - I can post pics if you prefer (I will have to reinstall SC4 first) but I think it's not needed.
- Streetlight colours can be changed at any time.
- Fenced GLR pack is actually based on this, the player can choose the black fences for a part of the city, then change the props and choose the red ones.
- Other packs using this feature is the Ped-Mall transit pack and Suburban GLR Stations, not to mention all packs with MML lots, where the player can install/uninstall the MML plugin; using an MML-controlled set of lots is not very practical, so some players may want to remove the MML plugin temporarily, make say, a waterfront using a set of walls, and then reinstall the MML to reduce the length of the menu.

So there are cases where DatPacking could cause inconvenience, and in such cases DatPacking should better be avoided. In some cases I warn in the Description and Readme: "DO NOT DATPACK THIS PLUGIN", but it's questinable how many players will read this, and remember it when they decide to datpack their plugins. It would be much better if this could be specified by the developer (using a text file, as I proposed in my previous post).

I hope I have convinced you.

[nerdly_dood]

Here is a list of things that I figure shouldn't be datpacked:
- RTMT
- All NAM Team products
- Terrain mods, rock mods and water mods
- Jeronij's and SFBT Street Side Mods
- Anything else that's modular, with interchangeable DAT files corresponding to aspects of the mod

What I think should be datpacked:
- Dependency packs
- BATs and lots that don't have interchangeable DAT files
- Mods that are in a single DAT file
- Mods that you install once and never intend to change

Also, from Wouanagaine's screenshot I get the feeling that this version is Vista-compatible... I'll have to try this!

[Gaston]

This is really great.   Thanks again Wou.   

Okay, now for a question.
     There are two "plugin" files.
     My document/ simcity/ plugins      and      Program files/maxis/simcity4/plugins    My question is,  What should go in the Program files/maxis/simcity4/plugins and not in the other?   I have a bunch of stuff in there but I can't remember how or when it got there.   


---Gaston

[Andreas]

Most, if not all custom content developers set their installers in a way that their work is installed into "My Documents". The Maxis landmarks are installed in "Program Files". For the game, it doesn't matter where the content is installed, since it reads from both folders - the one in "Program files" first and the one in "My Documents" afterwards. Using both folders can be interesting if there is more than one user account on the computer. Each account has a separate "My Documents" folder, so you can use the "Program files" folder for items that all users need for their regions, and things that are only used by a specific user can go into "My Documents". If you're the only user, it might be easier to use only one of the folders, since this is easier to manage in most cases.

[Gaston]

Thanks Andreas.    I have always wondered what the "progam file" plugins were for.    I seem to remember YEARS ago when we all were just starting this custom content stuff that some folks would have you put files in one or the other.   I was looking closer at it and notice most of the stuff is PEG stuff.    Hmmmm?   Thanks Again.   Sorry for the side track in the topic.   But it was sorta related.     


---Gaston