Hi folks
A new version of SC4DP has been uploaded to a LEX near you http://www.sc4devotion.com/csxlex/lex_filedesc.php?lotGET=26
(https://www.sc4devotion.com/forums/proxy.php?request=http%3A%2F%2Fimg355.imageshack.us%2Fimg355%2F330%2Fmainscreendpww4.jpg&hash=c0f8496ab0ff602536006f000a7c72759d2c37e7)
What's new
- This version is up to date with the latest finding in SC4 Loading order, faster, and provide some better compression which turns to speed up your loading time even more.
- You can now choose the folders you want to pack by selecting them in the mainscreen
Folder bigger than 392 Mb will be automatically split by SC4Datpacker, and will produce more than one mega pack, so no more need for manual folders splitting :)
- A new dependancy is needed, so make sure you read the readme.html to get the link
Some results
- 1.18 Gb of plugins packed in 2 min, loading time to region 35 sec, loading time to a large developped city 1min20
- 3.46 Gb of plugins packed in 5 min, loading time to region 55 sec, loading time to a blank city 1min15
- Rough estimation for a 3.6 Gb of plugins, without SC4Datpacker, 5 min loading time, with SC4Datpacker 2 min
This version has been tested intensively ( as you may notice given the letter of the build in the zip ) but of course I can't for sure tell it is 100% bug free, so please if you have some problem running it ( and after you check the readme for dependancies ) please post your problem here and if applicable post as well as the file that SC4Datpacker can't pack
Take care and enjoy fast loading time
ACE!!! another version. *installs windows just to test it out* $%Grinno$%
Joe
is this on the lex disk or do I have to download this?
It is available at the LEX (the old download has been updated).
Quote from: Andreas on November 27, 2008, 10:58:52 AM
It is available at the LEX (the old download has been updated).
so the lex disc version is outdated i take it?
So there is a new version! Great!
Any thought of implementing the feature I had proposed earlier? I mean making it possible to exclude some folders from DatPacking by putting a marker file in the folder, eg DO_NOT_DATPACK_ME.txt (no contents)? Yes, the user can specify which folder to DatPack and which not, but it would be good to let the developer decide this too, not just the user. I'm talking about RTMT, which has optional plugins, and these may be changed during the lifetime of the RTMT installation. I think it would be impractical to expect the players to remember which plugins should not be DatPacked.
Please consider implementing this feature in the next release or update (I think it's quite reasonable to reckon that this isn't much work).
Cogeo, I think you'll have a hard time explaining me why you don't want a user to pack a folder
I've just played with a 3.6gb all compressed, but the blgprop*.dat, even my working files are compressed, as well as your public RTMT version, without any problems.
If a player doesn't pack all its stuff, then he will have to think about where/what is loaded first, if it is needed to load stuff A in folder X before stuff B in folder Y which was packed in compressed plugins etc...
whereas using everything in the packed form, he just has to push the button and run the game, because if it works with all unpacked files, it will works with all packed files. That doesn't mean that if it works with all unpacked files, it will works with some packed and some unpacked files.
If anything change, like I useally change the pathfinding plugin in NAM depending of the kind of tile I'm playing, I just move in/out the correct file, and just run SC4DP and hit the 'start button. SC4DP only rebuild then NAM folder, that's it, nothing more, nothing to select or care about .
Great news for 2008 and also a new update for the anniversary of SC4D.... (Moved original post to the proper place....)
Here is my report about the new version...Its great!
My specs:
Plugins unpacked 3.80GB
Packed 3.62
Maxis plugins (NAM) .91.6MB
z folder & root files .54.6
Total plugins used 5.082GB
Comuter used: Dell 8400 Pentium 4 3GHz 2GB RAM
All files in both plugins folders are either datapacked or dat files, no model,lot,desc. Except for one
On first run I had 37.7MB of new stuff to pack in my various 21 folders that make up my original plugins.
It took around 5 minutes to do this and gave me a report with about 30 items I needed to remove and did so.
Ran again in about 1.5 minutes.
Settings used : Force repacking...NO: Optimize...Yes: Include TGI...Yes. Datapacker reported 2.68GB
My average load up time has dropped by 1 minute.
I played for around 2 hours in many cities (18sec between cities) everything seems to work will. :thumbsup:
Thank you for your great contribution to this game. &apls
This is probably a stupid question, but what is the DatPacker? What does it do?
To put in simply it converts your basic building file that is usually made up of a mode lfile, desc file, lot file into a dat file. And computers can read dat files a lot faster. As proof my game runs better now with 5GB of data packed plugins than it did when I only had 550MB of unpacked plugins. Hopefully there will be someone along that can give you a really technical answer.
Good non technical answer, High5Tower :thumbsup:
Technical answer
Every file that SC4 is reading is made up of 'entries' ( like afile in a zip file ). The job of DatPacker is just to read all your SC4 files, extract all those entries and group them in less files ( originaly one per subfolder in your plugins folder ). So instead of having 10000 files in 25 folders, you'll now have only 25 files.
As each entry is uniquely indentified, DatPacker takes care of getting the correct one when you have duplicates ( this can happen when a file is updated, and you didn't read the readme of that file asking you to remove older stuff for example, or when 2 mods modified the same entry ) SC4 only use one. So instead of SC4 making the correct choice at loading time, it is left to DatPacker, this will of course make a win when loading SC4, because SC4 will only read an entry once
In OS low level world, opening 10000 files is a very long operation, so now SC4 will only open 25 files, this is a big win
Chances are high that your 10000 files will be scattered all over your HD, whereas 25 big files are more likely to be in the same place ( especially if you run a Defragmenter ) so your HD will locate and read them much more quickly
Moreover, it has been proved by users that the zoom bug CTD ( ie zooming in/out very rapidly can leads to a crash to desktop ) is related to the number of files SC4 is trying to read to get the correct graphics, so with 25 files you don't have this bug anymore
thats amazing! I'm getting that right now &apls
Hi, how can download this program? the link on top opens a window without possibility to download (1 photo, many comment, nothing more :S )
thx
You need to be registered to the BSC LEX before you can download. It is a separate registration from the forums.
-Swamper
Wouanagaine, why is it so hard to believe that some folders should better be not datpacked?
In many cases the configuration of a pack is determined by the presence or not of certain plugins. And if the configuration can change during the lifetime of the installation, datpacking a folder (and having a single dat file) has two obvious disadvantages: a) the player is not able to tell if a certain plugin is installed or not, and b) removing a plugin that is merged into a common datfile is not possible (the player should delete the plugin and install it anew). In such cases having the option to skip a folder (and its subfolders) would be preferable.
And if you wonder if such plugins exist, NAM is the first that comes to mind. Surely, a DatPacked NAM will work, but then it's impossible to tell wheteher a plugin is installed or not, and it's also impossible to remove, say, turning lanes for roads. Therefore, to my opinion NAM should not be datpacked.
Many of my plugins can also be configured dynamically:
- RTMT props can be installed/uninstalled at will at any time allowing the props to be selected for each individual plop (eg design #1 along the main avenue, #2 at the suburbs and #3 in the industrial area); this is impossible if RTMT is datpacked - I can post pics if you prefer (I will have to reinstall SC4 first) but I think it's not needed.
- Streetlight colours can be changed at any time.
- Fenced GLR pack is actually based on this, the player can choose the black fences for a part of the city, then change the props and choose the red ones.
- Other packs using this feature is the Ped-Mall transit pack and Suburban GLR Stations, not to mention all packs with MML lots, where the player can install/uninstall the MML plugin; using an MML-controlled set of lots is not very practical, so some players may want to remove the MML plugin temporarily, make say, a waterfront using a set of walls, and then reinstall the MML to reduce the length of the menu.
So there are cases where DatPacking could cause inconvenience, and in such cases DatPacking should better be avoided. In some cases I warn in the Description and Readme: "DO NOT DATPACK THIS PLUGIN", but it's questinable how many players will read this, and remember it when they decide to datpack their plugins. It would be much better if this could be specified by the developer (using a text file, as I proposed in my previous post).
I hope I have convinced you.
Here is a list of things that I figure shouldn't be datpacked:
- RTMT
- All NAM Team products
- Terrain mods, rock mods and water mods
- Jeronij's and SFBT Street Side Mods
- Anything else that's modular, with interchangeable DAT files corresponding to aspects of the mod
What I think should be datpacked:
- Dependency packs
- BATs and lots that don't have interchangeable DAT files
- Mods that are in a single DAT file
- Mods that you install once and never intend to change
Also, from Wouanagaine's screenshot I get the feeling that this version is Vista-compatible... I'll have to try this!
This is really great. Thanks again Wou.
Okay, now for a question.
There are two "plugin" files.
My document/ simcity/ plugins and Program files/maxis/simcity4/plugins My question is, What should go in the Program files/maxis/simcity4/plugins and not in the other? I have a bunch of stuff in there but I can't remember how or when it got there. ()what()
---Gaston
Most, if not all custom content developers set their installers in a way that their work is installed into "My Documents". The Maxis landmarks are installed in "Program Files". For the game, it doesn't matter where the content is installed, since it reads from both folders - the one in "Program files" first and the one in "My Documents" afterwards. Using both folders can be interesting if there is more than one user account on the computer. Each account has a separate "My Documents" folder, so you can use the "Program files" folder for items that all users need for their regions, and things that are only used by a specific user can go into "My Documents". If you're the only user, it might be easier to use only one of the folders, since this is easier to manage in most cases.
Thanks Andreas. I have always wondered what the "progam file" plugins were for. I seem to remember YEARS ago when we all were just starting this custom content stuff that some folks would have you put files in one or the other. I was looking closer at it and notice most of the stuff is PEG stuff. Hmmmm? Thanks Again. Sorry for the side track in the topic. But it was sorta related. ;)
---Gaston
Quote from: cogeo on December 06, 2008, 11:28:49 AM
Wouanagaine, why is it so hard to believe that some folders should better be not datpacked?
Why it is so hard to believe that all folders should be datpacked ? :)
Quote
In many cases the configuration of a pack is determined by the presence or not of certain plugins. And if the configuration can change during the lifetime of the installation,
Do you think someone will pack his files during installation ??? ?
Once it is installed, it is installed
Quote
datpacking a folder (and having a single dat file) has two obvious disadvantages: a) the player is not able to tell if a certain plugin is installed or not
Only if the player is so stupid that he deleted his files instead of moving them out of his plugins folder, and I really won't take it by hand if he is so noob
Another option that I'm using, use the '-UserDir' option
Quote
and b) removing a plugin that is merged into a common datfile is not possible (the player should delete the plugin and install it anew).
And what is the difference with a not packed one ? he should remove it and install a new one. The only next step is to rerun datpacker which should be light as only one folder has changed
Quote
In such cases having the option to skip a folder (and its subfolders) would be preferable.
As I said, the user can specify which one he want to pack
Quote
And if you wonder if such plugins exist, NAM is the first that comes to mind. Surely, a DatPacked NAM will work, but then it's impossible to tell wheteher a plugin is installed or not, and it's also impossible to remove, say, turning lanes for roads. Therefore, to my opinion NAM should not be datpacked.
So lets say a player pack everything but NAM he will have in his final plugins folder
\Compressed_plugins
\NAM
Now what happened if in his original structure he had
\A
\B
\NAM
\OtherStuff
\ShouldLoadedAfterNam
the result will be that files in \ShouldLoadedAfterNam will be packed and put into \Compressed_plugins\ShouldLoadedAfterNam.dat with A.dat B.dat OtherStuff.dat
So the end result will be that it won't work anymore because \Compressed_plugins\ShouldLoadedAfterNam.dat will be loaded before the NAM instead of after
And I want to guarantee that the game will work exactly the same with everything packed compared to nothing packed, because if I can't guarantee that I'll have a freaking numbers of complains
Quote
Many of my plugins can also be configured dynamically:
- RTMT props can be installed/uninstalled at will at any time allowing the props to be selected for each individual plop (eg design #1 along the main avenue, #2 at the suburbs and #3 in the industrial area); this is impossible if RTMT is datpacked - I can post pics if you prefer (I will have to reinstall SC4 first) but I think it's not needed.
- Streetlight colours can be changed at any time.
- Fenced GLR pack is actually based on this, the player can choose the black fences for a part of the city, then change the props and choose the red ones.
- Other packs using this feature is the Ped-Mall transit pack and Suburban GLR Stations, not to mention all packs with MML lots, where the player can install/uninstall the MML plugin; using an MML-controlled set of lots is not very practical, so some players may want to remove the MML plugin temporarily, make say, a waterfront using a set of walls, and then reinstall the MML to reduce the length of the menu.
I do that everyday ( the MML thing ) and I pack everyday, is it so hard to add/move/remove files from original structure and repack ???
Quote
So there are cases where DatPacking could cause inconvenience, and in such cases DatPacking should better be avoided. In some cases I warn in the Description and Readme: "DO NOT DATPACK THIS PLUGIN", but it's questinable how many players will read this, and remember it when they decide to datpack their plugins. It would be much better if this could be specified by the developer (using a text file, as I proposed in my previous post).
I really don't care of people not reading readme, they can blame you, but in the very end, it is their fault
Quote
I hope I have convinced you.
I don't think, even if you raise good points, I don't want to sound bad or flaming, but as I said my 1st priority is to make the game run the same with or without SC4DP.
Quote from: nerdly_dood on December 06, 2008, 01:22:02 PM
Also, from Wouanagaine's screenshot I get the feeling that this version is Vista-compatible... I'll have to try this!
Yes developed under Vista ( 32 bit, Basic version )
Hey Steph,
I also believe its up to the player to decide rather to pack or not to pack a certain folder if they so choose, heck I do that!!! I got all my dependencies folder for everything IE BSC prop pack to mega to textures are not dat packed at all... I also haven't dat packed bsc lot files why? Its a personal preferences I guess, or its because I'm DL'ing so much wonderful BSC creations its hard to keep up and why bother lol....
But I also have to add I keep all of my plugins folders that I had datpacked, I call that "uncompressed folder December" in that folder is all the folders that are the original folders that wasn't datpacked... Of course the Datpacked files are now in my plugins folder that I play with... So that way I can keep up with any changes I need to in airport lot files or NHP Lot's and misc lotters and so on... But that is my point on it...
hey i cant download SC4DatPacker 2008 :S how ?