What causes Prop Pox (and how to avoid it)

[Turjan]

I just can't ignore the one nagging fact, no-one in 6 years has found an instance of this problem that could be tracked down.

So, you don't believe the first four posts in this thread are correct, or do you mean anyone except that case?

[epicblunder]

Normally i don't comment on this subject, but i feel the need to point something out here for everyone. 

At the moment, I'm thinking of backing out from using seasonal trees. Not in the tree controller - that one doesn't touch the original Maxis trees - but in tree replacers. I have absolutely no idea what caused the prop pox in my case, but I may look at that mod that replaced trees on Maxis lots with random seasonal trees. I'm not calling it out, as I have no idea whether this mod is related to my issues. In any case, given the number of mods used, it's like looking for a needle in a haystack.

Tree models can be placed into the game in 2 ways: as a flora exemplar or a prop exemplar.  Tree controllers and tree replacers use flora exemplars, which DO NOT write to the network subfile; instead they write to a subfile with Type ID a9c05c85 (not 2990c1bc).  So while it is possible for flora to bloat the city's filesize, it does not bloat the network subfile which everyone worries about.  I tested this thoroughly when making my last tree controller because with using HD flora on tight spacings it is possible to create absurdly large filesizes in some situations.  At no point in my testing did such cities contract "prop pox", and that was with exaggerated cases that in no way would be possible to recreate with currently available controllers.  The only way a tree or bush or flower gets put into the game as a prop is when someone adds it as a prop to a lot.

(*sidenote:  Tree controllers actually DO contain copies of maxis flora such as the maxis pines, oak, and scrub brush, but we don't bother touching the RKTs.  We need to overwrite the "Flora Table" in the maxis files so the game engine doesn't try to place them with the God Mode tool)

AFA the seasonal trees on maxis lots mod:  If it's the one i'm thinking of, that mod contains relots of all the maxis files with the old trees deleted from lots and new trees manually placed on them, it is not a mod that that overwrites maxis tree RKTs with new RKTs.  A quick look inside the mod reveals only .sc4lot files, no .dats with RKT overwrites.  I've used it myself since its release with no problems.

---

I also want to point out that i think there are quite a few mis-diagnoses of custom content causing "prop pox".  The only time any of my cities contracted the pox, it was actually file corruption because the client CTD'ed mid-save thanks to my antivirus starting a scheduled scan (and it was a skyscraper city; no RKT4 overwrites there.  RIP Stormwatch, Kaui).  When people's props start disappearing they jump around hell bent on finding a plugin to blame.  But i wonder: is it a plugin or is there file corruption from this rickety old .exe trying to write to the hard drive while the user has too much crap running in the background (or, gods-forbid, someone tried to alt-tab durring a save).

[Turjan]

Yes, I hoped to make clear that the tree controller wasn't to blame. I learned my lesson and did not remove any trees that I used, ever. That region went through 3 tree controllers, and all trees are still there as I made sure to keep MMPs of them (even the Maxis ones).

I also addressed the relots with seasonal trees in my third post. Removing that mod is definitely not the issue in my case, as I looked, and the props are still there after removal of the mod. I should probably edit my second post.

I also want to point out that i think there are quite a few mis-diagnoses of custom content causing "prop pox".  The only time any of my cities contracted the pox, it was actually file corruption because the client CTD'ed mid-save thanks to my antivirus starting a scheduled scan (and it was a skyscraper city; no RKT4 overwrites there.  RIP Stormwatch, Kaui).

This may well be the case here. I hope it was clear that I did not blame the SPAM in any way or form. First, I removed this content without deleting all instances first (a big no-no). And still, the other tile with even more SPAM fields managed to get rid of all disabled props, somehow. The poxed city didn't, for whatever reason. I should probably try my luck and see whether I can ruin the other tile .

Edit: Test was "positive". City got prop poxed. Still with 0 disabled props.

[joshua43214]

Well I was a little offended by the comments of joshua43214, it's a slur on the reputation of creators, that comes across with a whiff of elitism and totally disproportionate (picking up a theme here?). I am not bent out of shape to my knowledge, the simple truth is that most content does work and won't break your game. So to suggest the only way to be safe is to ignore the vast vaults of every other file exchange as somehow prudent is frankly ridiculous.

I stand by my words in this matter, and the BDK file is the perfect example. Peg updated the file on his own site, but did not bother to update the file on the STEX. So there it sits, waiting to ruin regions. The biggest strength of the STEX is also it's greatest weakness. I learned long ago to always closely examine STEX downloads before I let them loose in my game. I have read this entire thread in the past. As a research scientist, I accept the method used by the OP to determine the file that was causing prop pox on his machine. I accept the change from an RKT 1 to an RKT 4 as a good theory for what causes the pox. The game clearly re-examines the savegame  when it reaches a certain size threshold, and during this process the pox strikes.

My intent was not to slur the creator that upload to the STEX, I have many STEX bats and stuff. My intent is always to warn, never get something from the STEX, if you can get it from the creators own website, and when you do get something, look it over closely.

Anyway...

I modified a piece of code that I wrote for opening and scanning dat files.
It first scans the SimCity_1.dat file and creates a dictionary of all props and their RKT's. It then scans all the other .dat and .sc4desc files, when it finds a new TGI, it adds it to the dictionary. When it encounters a TGI already in use, it compares the RKT's between the two files. If the RKT's are different, it adds both files to a new dictionary of potentially bad dats. Lastly, it writes tab delimited .txt files for the SimCity_1 props list, the complete prop list for all dats, and lastly a bad dat file that has the the names of the files in question and their respective RKT's.
The nice thing about the script, is that it compares every TGI to the others, not just to the SimCity_1.dat on the assumption that creators might be overwriting each other (which I did find).

Code Select Expand

['0x6534284a', '0x8fb08a82', '0x10d16eb7', 'RKT4', 'bsc mega props - misc vol02.dat', '"rtprop_snm_paratrooper"', 'RKT1', 'bscbatprops rt snm vol01.dat', '"rtprop_snm_paratrooper"']
['0x6534284a', '0xc977c536', '0x29110000', 'RKT1', 'SimCity_1.dat', 'R1x2x2_Recliner_2911', 'RKT4', 'peg-oww2_bdk_resource.dat', '0x20\tr1x2x2_recliner_2911']
['0x6534284a', '0xc8dbccba', '0x0c9db0fe', 'RKT0', 'pmct_lots.dat', '"pmc_prop_centrepole"', 'RKT3', 'pmct_namv061705.dat', '"pmc_prop_centrepole"']
['0x6534284a', '0xc8dbccba', '0x0c9db0fd', 'RKT0', 'pmct_lots.dat', '"pmc_prop_sandstone"', 'RKT3', 'pmct_namv061705.dat', '"pmc_prop_sandstone"']
['0x6534284a', '0xc8dbccba', '0x0c9db0fa', 'RKT0', 'pmct_lots.dat', '"pmc_prop_openpaved"', 'RKT3', 'pmct_namv061705.dat', '"pmc_prop_openpaved"']
['0x6534284a', '0xc977c536', '0x29b20000', 'RKT1', 'SimCity_1.dat', 'R1x1x2_BeachChair_29B2', 'RKT4', 'peg-oww2_bdk_resource.dat', '0x20\tr1x1x2_beachchair_29b2']
['0x6534284a', '0xc8dbccba', '0x0c9db0fc', 'RKT0', 'pmct_lots.dat', '"pmc_prop_pavement"', 'RKT3', 'pmct_namv061705.dat', '"pmc_prop_pavement"']
['0x6534284a', '0xc8dbccba', '0x0c9db0fb', 'RKT0', 'pmct_lots.dat', '"pmc_prop_cobblestone"', 'RKT3', 'pmct_namv061705.dat', '"pmc_prop_cobblestone"']
['0x6534284a', '0xc977c536', '0x290d0000', 'RKT1', 'SimCity_1.dat', 'R1x1x3_PatioChair_290D', 'RKT4', 'peg-oww2_bdk_resource.dat', '0x20\tr1x1x3_patiochair_290d']

The first three entries are the TGI
The next three entries are the file being over ridden
The last three entries are the over-ridding file
The BDK file is there as a control

Note the BSC files being changed from 1 to 4. I have no idea if this is a problem or not, but snm or the pmct might be a cause of unexplained prop pox. pmct is the PedMallCompatibleTransit pack (from the STEX    )

I need a bit of time to wrap the script in a GUI (I kinda suck at GUI), and convert it to an .exe.
If any one is interested in testing the tool, send me a PM and I will give you a copy of it. Once I feel like it will run ok on other machines, I will upload it to the STEX  for everyone to enjoy.

Out of 17471 props scanned, it is impressive that this is all the files that are potentially bad in my plugins folder.

[vortext]

That's a very promising tool, nice job. 

As for the issues you found, that'd need to be determined by closer inspection; i.e. does the RKT4 file actually have timing properties. As I recall changing from RKT1 to RKT4 in and of itself doesn't matter. What matters is whether or not the RKT4 prop is timed since it's the additional info pertaining to timing that gets stored in the savegame and later on causes the discrepancy, i.e. corruption when it gets disabled. Instead RKT4 may have been used to offset the prop, since apparently it's a paratrooper, which afaik isn't an issue.

Not sure about the RKT0 & RKT3, both typically use the same kind of model (a single S3D used for all rotations & zooms) and iirc RKT3 is only ever used for transit exemplars (overhanging models and such) so I suspect it's not saved in the prop subfile to begin with.

[nas786]

I tried doing the graphic trick to limp forward...but it must require saving and exiting, etc...and that would just exasperate the problem.

You may as well kill the tile...and any other large tiles you played and saved while the BDK file was in your plugins.

Hey, so I'm doing the graphic trick currently, what happened exactly as you kept moving forward with that?  Because as I said, I don't mind playing w/ missing props vast majority of the time, as I stay busy building stuff rather than enjoying the scenery.  And then every now n then I wanna enjoy the scenery, I do that trick.  Is this going to lead to anything worse down the road?

Also I should point out, I don't have the BDK file, I have actually never downloaded it ever, so it def doesn't exist anywhere on my computer, or hard drives, or diff plugin folders.

[nas786]

@NAS786

Plenty of props (and other things) get overriden, there is nothing inherently wrong with doing this. If the files in PEG Random Woods do not override other game-exemplars then again they too can not cause prop pox. Many files have been listed in this thread as "potential suspects" I will repeat than in all the years this thread has been going not one single new file with this problem has been found, not one, don't worry about it, enjoy your game, you could get hit by a bus, but you probably won't - I'm sure you don't spend much time contemplating how to avoid it either way.

Yea, I get what you saying bro, I just been moving forward with the graphic trick, and I'm good with it.  To be honest, I was going through all that stuff more for the sake of our community at large, know what I mean.  for the cause, homie.  Figured if I could identify something, it may help others avoid it.  But as far as myself, I'm a lazy bastard, soon as I found the graphics trick I was good to go! lol

[mgb204]

So, you don't believe the first four posts in this thread are correct, or do you mean anyone except that case?

I think that anybody spending 10 minutes of their time to check back through a couple pages could answer that question for themselves?

---

However the law of probability is against you here from every angle. The chances of getting prop pox, the chances of finding what caused it, the probability of all these things is so close to zero that beyond taking a few precautions to protect yourself, there really isn't anything you can do about it. That is my entire point here, if you want to spend time tracking the problem down, by all means do that. However I think taking all the facts into consideration that it wouldn't be in someones best interest to advise them to invest a lot of time tracking it down, especially if you don't really have the knowledge of the game to understand what you are looking at it whilst doing so.

So in short, my advice (opinion) would be to accept the files are toast and simply move on. You may choose to limp on with the region using various tricks to keep your cities going, but the results of that are played out here too, eventually it will be too broken to continue. If sometimes I come across as trying too hard to find a diagnosis that is not prop pox, that's simply because many cases are not prox pox and misdiagnoses here has some pretty catastrophic effects. So when I ask "are you sure it's prop pox" my intentions are to be certain in my mind that you have correctly diagnosed the problem, because if you haven't then almost certainly the people on these forums would be able to help you to get your cities back and running. The last thing I'd want to do is be cavalier in my assessment of a problem like this, that'd be like a vet looking for excuses to put your pets down, rather than trying everything possible first to avoid such a scenario.

[epicblunder]

Yes, I hoped to make clear that the tree controller wasn't to blame...

Aye.  But as mgb said, once even a hint of suspicion comes up people tend to shy away from things just to be safe.  I wanted to clarify for everyone (not just you) that these type of files absolutely don't act in the way that people suspect is the cause of prop pox. 

[Turjan]

So in short, my advice (opinion) would be to accept the files are toast and simply move on. You may choose to limp on with the region using various tricks to keep your cities going, but the results of that are played out here too, eventually it will be too broken to continue. If sometimes I come across as trying too hard to find a diagnosis that is not prop pox, that's simply because many cases are not prox pox and misdiagnoses here has some pretty catastrophic effects. So when I ask "are you sure it's prop pox" my intentions are to be certain in my mind that you have correctly diagnosed the problem, because if you haven't then almost certainly the people on these forums would be able to help you to get your cities back and running.

Well, SC4 Save claims it's prop pox, and it definitely looks like it. Regarding the rest, no worries. I had made a backup of the whole region and the whole plugin configuration I used in that region before I moved the files out. The prop pox struck immediately after I had moved the plugins out, on my second play session after this (when I got to a tile with the necessary file size). Which means I just lost a few hours of meddling around, which is nothing. I can just revert to the state from before.

Aye.  But as mgb said, once even a hint of suspicion comes up people tend to shy away from things just to be safe.  I wanted to clarify for everyone (not just you) that these type of files absolutely don't act in the way that people suspect is the cause of prop pox.

That's probably a good thing to do. I never had any problems with tree controllers, except the occasional CTD when I didn't keep MMPs from the controller that wasn't used anymore in the very beginning (there were some ghost Maxis trees left) when I didn't know any better.

[cogeo]

Hi all,

Joshua asked me per PM about the prop-pox and my PMCT (Pedmall-compatible Transit Pack), obviously because of the RKT0 to RKT3 overrides reported above. My answer here is that there should be no problem, as both prop exemplars are simple (ie not timed or seasonal). The RKTx properties are not saved in the city file, instead they are referenced (in the effective exemplar). Besides, the RKT3 version here overrides the RKT0 one fully, that is all props will be recorded using either version of the exemplar, and then red having the same effective definition; it's an intra-package override, not an Original-by-Plugin one. The RKT3 version is for the newer versions of NAM.

Not sure if it has become clear of what really causes the prop-pox. My understanding is that it is caused by redefining a simple prop exemplar (usually in SimCity_1.dat) as a timed or seasonal one (or the opposite), because for the latter a longer record is written in the city file for each prop instance (it is a little longer because it stores a little more information, like state or days-elapsed-since-activation or who knows what). So if the length of the record stored in the city file does not match the one implied by the exemplar (based on the presence or absence of seasonal or timed properties) the program is somehow "lost" and cannot read the props' data correctly (this is my understanding). Props using the NightTimeChange property (in combination with RKT4) are NOT really timed, as there is no prop "state" (simply a different model is displayed during nighttime). And as mentioned above I'm quite sure that the RKTx properties are not saved in the city file, so you are free to override them with no problem. Accordingly, the program should rather be modified, so as to check whether a simple prop was redefined as a seasonal/timed one (or the reverse), instead of comparing whether the RKTx property was changed to another one, or even removed completely (or reversely, added).

All the Best

[joshua43214]

Hi all,

Joshua asked me per PM about the prop-pox and my PMCT (Pedmall-compatible Transit Pack), obviously because of the RKT0 to RKT3 overrides reported above. My answer here is that there should be no problem...

Accordingly, the program should rather be modified, so as to check whether a simple prop was redefined as a seasonal/timed one (or the reverse), instead of comparing whether the RKTx property was changed to another one, or even removed completely (or reversely, added).

All the Best

Thank you very much for responding.

My understanding is about exactly the same as what you are saying. The issue is changing a prop with a small memory footprint to one with a larger footprint.
I have debated in my mind your suggestion about modifying the program since I started writing it. On one hand, it could be unfair to folks like you that are on my short list of trusted authors. On the other hand, I hate withholding information that might be valuable. I have no intention of removing the PMCT from my own plugins, I don't want to be responsible for possibly scaring folks away from an excellent mod. The Libertarian in me says "just give folks the information, and let them make a grown up decision."

I finally got the GUI working (except the cancel key), the .exe runs on my system, and I need to prepare a readme. Going to test it on another PC before I send it out for limited testing. I will decide how to handle the issue before it goes public. Ideally, the output would warn the user that a particular RKT change is safe or dangerous. Unfortunately the output goes to a txt file meant for use in Excel or Notpad++, and there is not good method for formatting such a warning. I am definitely open to suggestions on this subject.

This script originally started as a vision for a dependency finder. The idea was to write a program that would create a list of every TGI in a mod file, give the program out to folks with really big plugins folders, and have them send their files back to me. I would then merge the files into a master list, and release the program publicly. Users could then upload their TGI list to a place I could access it, and then I would periodically release an updated master file. I bet that 10 users with a lot of files on their system could cover >90% of the TGI's. Used in conjunction with SC4DataNode, players would be able to find what dat and missing dependency comes from. This could be a really good tool used for getting rid of brown box's and finding missing textures.

Writing the prop checker actually helped me solve a couple of problems I faced with the original concept. I am currently considering holding the tool back and finishing the original concept with the RTK scan as an optional scan. The real issue here is that I am a molecular geneticist and a computational biologist, not a developer. I write very programmatic code that just takes large data files, and make pretty graphs from the stats. I kinda suck at writing GUI's, and I am not even sure where to start with making the program able to auto-update.

-Josh

[catty]

...The idea was to write a program that would create a list of every TGI in a mod file, give the program out to folks with really big plugins folders, and have them send their files back to me. I would then merge the files into a master list, and release the program publicly....

wouanagaine wrote a program that would go thru all your LOTs and generate a html file file with that kind of information in it, unfortunately it was never released, you used to be able to download it from the first post in this topic

http://sc4devotion.com/forums/index.php?topic=86.0

link no longer works   

[joshua43214]

...The idea was to write a program that would create a list of every TGI in a mod file, give the program out to folks with really big plugins folders, and have them send their files back to me. I would then merge the files into a master list, and release the program publicly....

wouanagaine wrote a program that would go thru all your LOTs and generate a html file file with that kind of information in it, unfortunately it was never released, you used to be able to download it from the first post in this topic

http://sc4devotion.com/forums/index.php?topic=86.0

link no longer works   

My idea is a bit closer to what DataNode does. You would be able to scan a file, and it would return all the TGI dependencies. It would then compare the required dependencies against a crowd sourced master list of TGI's, and return what file contained the missing TGI. You could then Google up the missing file.

[catty]

A few years ago I used SC4LotInfosGenerator and scanned my plugins, the LEX DVD and CD and the STEX Disks (2, 3 and 4) ending up with a huge number of duplicates records obviously and as I only wanted to know the name of the plugin and its file name I ended up with an Microsoft ACCESS database containing this information

Code Select Expand

GALLERY File Name
0xEE668F70 SG_Lots_Motels.dat CS$$4_3x3_SG_BrewPubFishTaleAle
0xEE668F70 CS$$3xSG_BrewPubFishTaleAle_ee668f70.SC4Lot CS$$3xSG_BrewPubFishTaleAle
0xEE66901B SG_Lots_Motels.dat CS$$2_2x5_FishTaleInn
0xEE66901B CS$$2xFishTaleInn_ee66901b.SC4Lot CS$$2xFishTaleInn

57215 lines in total, I've exported it out as a text file (its still 4mb in size) and put it "dino's" briefcase over at CB in case anyone wants it.

The Gallery ID is from the plugins TGI, but it also helps me identify the picture of the plugin if I want to check I'm looking at the right item (only 18717 pictures    )

I'm attaching a picture from my files

Cathy

0x0C2C1272   Microwave Power Plant_0c2c1272.SC4Lot      Microwave Power Plant

[joshua43214]

To anyone spending their Sunday looking for RTK changes in their plugins folder, don't bother.

I have been doing some further experimentation and I can say categorically, with no reservation, that changing RKT1 to RKT4 as in the BDK file is NOT the cause of prop pox.
The issue lies elsewhere in the exemplar file.

My GF is on her way over for a delayed Valentines Day (weather...), so I do not have time to give a full write up, and I have some further testing to try and narrow the cause down further.

-Josh

[HappyDays]

Intriguing. We await your analysis.

[joshua43214]

Sorry for the delay.
This will be a long post. I will try to keep the most important stuff at the beginning.
tldr: changing from RKT1 to RKT4 does not cause the pox.

Something about prop pox had me bothered, especially after reading Cogeo's response above. Something just felt contradictory to me.
If changing the RKT from a short to long on a prop caused it to overwrite good data on a save, then the PMCT would cause prop pox. However, I have faith in Cogeo, and have no real reason to believe that his PMCT caused prop pox.
Also, if changing the RKT caused prop pox, then the BDK umbrella should not cause the pox since it does not change the RKT.

I decided to do some testing of my own to see what I could come up with.
Bap's original cities he uploaded have vanished along with fileden.com, so I decided to create my own test cities.

I found that it is painfully easy to make a city with prop pox.

A city ready to pox, only has about 61,000 sims. all living in LD residential.

The prop file in the save game will toggle from compressed to uncompressed when the new zoning is about 2/3 done.

One of the reasons this issue concerns me so much is that I build a lot of slums arround my dirty industry, and I have been a victim of the pox a number of times even though I stopped using the BKD years ago.

I used a Plugins folder empty of any mod except the extra-cheats dll. I had to install it for the money cheat until the city developed.
For positive and negative controls, I grew the city till it was done developing with and with out the BDK installed.
With out BDK, no pox
With BDK, pox
This indicates my test city works for purposes of testing.
Unless specifically noted, I ran each test until the city finished development, saved the game, and checked the save game in the SC4Savegame Explorer
http://sc4devotion.com/csxlex/lex_filedesc.php?lotGET=2021
Because of the way this city was made, toggling between compressed and uncompressed prop file is as simple as zoning or dezoning.
I reloaded the save from a back up between each test.

I decided to begin with the PMCT, this is the great Ped Mall Compatible Transit pack by Cogeo from the STEX
http://community.simtropolis.com/files/file/13147-pedmall-compatible-transit-pack/
This version is meant for the NAM, which I did not have installed.
To be safe, I plopped 80 of the centerpole plops (lots of brown boxes).
No pox
I plopped 200 more, ran the game for a while
no pox
dezoned until the prop file compressed
no pox
rezoned and ran til prop file un-compressed
no pox

The indication is that the PMCT does not cause the pox.
This also suggests that changing the RKT from 0 to 3 (a small RTK to a big RTK) does not cause the pox.

I next turned my sights on the BDK.
I compared the BDK files for the Umbrella, recliner, beach chair, and patio chair to the game files in SimCity_1.DAT.
The Umbrella had the "Nighttime State Change" (SimCity_1) replaced with "Prop Time of Day." This is the only change to the Umbrella.
The recliner and chairs all had the same changes. The RKT1 was changed to RKT4, and a "Prop Time of Day" property was added.

I began by deleting the Umbrella from the BDK, added the BDK to my Plugins, and ran the city.
it did not pox.
I added some amenities and created a mid-wealth area with some high-wealth residents.
it did not pox.
Puzzled, I wondered if there was none of these props, or possibly not enough to cause a problem. SC4Save does not tell you how many of a prop are in the save, but it does provide a line for each instance of a given prop.
each page on SC4Save displays 32 lines, I counted 141 pages for the beach chair, 129 pages for the patio chair, and 131 pages for the recliner.
I could not verify that the recliner, the beach chair, or the patio chair cause the pox.

I next deleted the chairs and the recliner from a fresh BDK copy (has the umbrella)
It poxed right away.

I then deleted the chairs and recliner from a fresh BDK, and deleted the "time of day" property
it poxed

I took a fresh BDK, and replaced the "time of day" with a "nighttime state change" with no values
it poxed

I took a fresh BDK, and replaced the "time of day" with a "nighttime state change" copied from SimCity_1.DAT
no pox
I changed the "nighttime state of change" from the default 0x01 to 0x00 in the same file, and reloaded the un-poxed master file
it poxed

Then I took a fresh BDK and simply copied the "nighttime state of change" property from SimCity_1 to the BDK
no pox

as a control, I deleted the umbrella from the BDK, and added a "Self illuminated" property to the chairs and recliner
no pox
I added the "self Illuminated" and "nighttime state of change" property to a fresh BDK
no pox

The indication here is
the beach chair, patio chair, and recliner do not cause the pox
changing from RKT1 to RTK 4 do not cause the pox
deleting the "nighttime state of change" property from an RKT4 causes the pox.
Adding a "time of day" property to a RKT4 does not cause the pox

At this point I went back and re-read all 33 pages of this thread to see if anyone had looked at this before.
I did some more "extreme" testing, details are below. I can actually demonstrate that on its own, the BDK umbrella does not cause the pox. My contention is the pox occurs after the prop has been removed or changed like when a property develops or is redeveloped. In a real game, it causes the pox. See the end for details.

I need to vent a bit here.
In my field (genetics and computational biology), people that falsify data or claim to have run experiments that they did not run are considered the lowest of the low. There are examples of this that have caused havoc for years and cost the lives of thousands of people because important treatments did not reach the market, or bad treatments did.
The total amount of time to create my test zone and edit the BDK was well under 1 1/2 hours. I have no compunction about naming a person who claims they "conducted an investigation and found no problem" a bold faced liar. To then turn around an denigrate the extremely hard work of people investigating this problem, and claim that the issue is "invented," "caused by the NAM," "furthering the dark aims of the BSC," or "caused by random chance" is not only a liar, but a detriment to the whole community. All the beautiful work not withstanding, we would be better off never to have this person in our community.

I would once again like to formally tip my hat to Bap, wounagaine, Ripplejet, Barbyw, andreas, z, Cogeo, and all the others that contributed to this thread, and tried to not only find a solution, but tried pave a road to peace with the SimPeg website at the cost of permanent ban.

OK, back on topic.
When performing research. One of the things you never do is change your method of measuring or change the tool used to measure results. Something as simple as using two different microscopes can prevent funding and publication.
Bap did his experiments the "hard" way. He had to use a shrinking un-compressed prop file to indicate prop pox.
The problem with this method is that it is confounded when mid-wealth and high-wealth push out the low wealth. There are fewer props in a high-wealth neighborhood than a low.
I could not find a report of anyone checking just the chairs and the recliner after wou published the SC4Savegame Explorer.
I believe this is the reason that the recliner and chairs were blamed for the pox.
I should be really clear here, negative evidence is not evidence. Just because I did not see the problem, does not mean that it does not exist. There might be another factor here. The most obvious one is that my game save file is no where near as big as Bap's was. It is possible that there is some secondary issue when another file is forced to decompress.
It seems more like to me that Bap's method was confounded, than my method was confounded by a secondary cause.

If anyone has a copy of Bap's test cities, it would be great if you could verify that the chair and recliner do or do not cause the pox.

The key piece of information here is "what exactly is in the savegame?"
The key question here is "what exactly goes wrong with this information when the save is un-compressed?"
There was plenty of speculation, and Ripplejet posted some tentative results from this very query.

I spent some un-happy time with the information on this page
http://www.wiki.sc4devotion.com/index.php?title=Prop_Subfile
trying to write a script to read the prop file so I could monitor changes to it. Sadly the information on the page produces a format that is very different from the format of my savegames. It will take a great deal of trial and error to make it read properly, though I have made some progress.
Being able to read the prop file is critical to demonstrating proof of the cause of prop pox.

Until I can read the file, I have a theory, and some further evidence.
When the game loads, as we all know, it loads all the files in a certain order, it then does whatever the last files loaded says to do.
What the game does not do is go back and "forget" obsolete files, it just holds everything in memory.

The Theory:
When the game saves, it goes through every file in memory in loading order. It creates a field in the SGProp (I believe) for every piece of information required to load the prop. When it reaches the last prop in load order, that information overwrites all the information already entered. In the case of the BDK umbrella, the "nighttime state of change" field is added by SimCity_1 dat, then that field is null in the BDK.

A short explanation of how the DPBF file format works is required to explain how this causes a problem.
The BPBF format is a proprietary database format created by EA for the Sim games. Just about everything in the game from models to images are stored inside a DPBF file. The files we are talking about in this discussion are "exemplar" files (whatever the heck an exemplar is).
After the exemplar has been de-compress, and converted from binary into readable text, it contains a series of lines.
The first three lines always contain the same information in the same order: file code, parent cohort, property count.
After the third line, each line has two entries, the first being its "tag" ("Name value" in the Reader), and the second being the value.
Below is the umbrella from the BDK

Code Select Expand

EQZB1###
0,0,0
14
0x10 30
0x20 R2x3x2_$$Beachumbrella_2900
0xabfc024 28
0x27812810 ['0x40000000', '0x40000000', '0x40400000']
0x27812824 ['0x0', '0x0', '0x0', '0x0', '0x27812821', '0x5ad0e817', '0xbadb57f1', '0x29000000', '0x1', '0x0', '0x0', '0x0', '0x27812820', '0x5ad0e817', '0xbadb57f1', '0x0']
0x27812870 ['0xa0000001']
0x49a1e05a 1235347861
0x49c9c93c 1
0x4a149631 ['0x41100000', '0x41900000']
0x4a89fcf3 True
0x6a95e503 False
0x8a416a99 ['0x2026960b', '0x6a554afd', '0x6a8c223b']
0x8a5e5db8 True
0xe9a316eb 2
The game reads the exemplar, see that it is a binary that needs to be decoded (EQZB), it has no parent cohort (0,0,0), and has 14 properties. If it has to many or to few properties things go wrong.
the tag 0x27812824 tells the game this has a Resource Key Type 4 (RKT4), and the 16 values on the left are basically the what/when/where information about the prop.
The tag 0x4a149631 is the "time of day" property, the numbers on the left are a hex version of 9am to 6pm.

So in essence, the game reads the file, sees it has 14 properties, goes through each property and does the correct thing. This is a very robust system, the file tells the game what type of information it has, then it tells the game what that information is.
The save game is not so robust.
Rather than a series of ordered pairs (tag,value) it is just a series of values.

below is a partly decoded save file.
The tags on the left are NOT part of the file. I added those so I could read it.
The file is just a series of values.

Code Select Expand

size 181
CRC û
mem 0x29e201c
maj_v 0x6
min_v 0x4
zot 0x0
unk_1 0x0
ap_flag 0x0
something 0xa823821e
min_tract_x 0x47
min_tract_z 0x47
max_tract_x 0x47
max_tract_z 0x47
x_tract_size 0x2
z_tract_size 0x2
prop_count 0x0
TID 0x6534284a
GID 0xc977c536
IID 0x29110000
instance_ID 0x43fbd3d2
min_x 3.21964677141e-14
min_y 503.654846191
min_z 270.0
max_x 502.793548584
max_y 504.654846191
max_z 272.0
orientation 0x65
state 0xfc
start_time 0x43
stop_time 0x0
count 0x0
It clearly has a decoding error around the Instance ID, since the min_x value is clearly wrong, but it is in frame up to the TGI values (this is the recliner).

I believe that the game creates a field for the Nighttime State Change, and the BDK enters a null value.
When the file is compressed, tags are added giving the compressed and uncompressed size of file.
The game is not able to understand the null value, it looks for x number of properties, but finds x-1. It thinks the Nighttime State Change property is the nth property, but it is missing, so part of the next property is read instead. The error then propagates forward to the end of that property.
The game is not able to compress a null value.
Somewhere during the process of decompression and compression, the game gets confused about the expected file size and number of properties, and the actual file size and number of properties.
If things go wrong, it will write the file forward into space reserved for another prop. This prop is now corrupted, and the process continues.

Since I could not verify this to be the case, it will have to remain a theory until someone can help me out with the save format, or I figure it out myself.

As a further test, I made this.

It is an empty grass lot, covered in about 750 umbrellas.
I plopped about 3 1/2 rows from end to end on a small city tile until the props folder was at is max compressed size.
Sorry, no screenie, it crashes the game to look at them lol.
I then added the BDK file, plopped 5 more of these so the file uncompressed.
no pox
ran it through and entire day night cycle.
no pox.
It seems the pox can only occur during development as properties are created and recreated.
This is only supporting evidence of my theory. We will have to wait until the save file is more fully decoded for proof.

I am brainstorming on how to fix my Pox scanner to deal with this. I am not sure if load order is a factor here or not. It seems to me that if the BDK loaded before the SimCity_1.DAT, that the same problem would occur, only this time SimCity_1 would be the cause of the pox. Checking this level of data means that every exemplar will have to be decompressed, and a rather large check file created and passed over. My current scrip will scan my 3.7Gb test folder, open 1649 files, and compare RTK's on 18,313 props in about a minute, and is written in pure Python. Python can certainly do a deeper search, but this is really moving into something that should be programed in C (which I can't write), especially if load order is important.

-Josh

[HappyDays]

Edit: I conflated "null" with "0". My post has been corrected. Nighttime State Change is a binary. 0 sets it to "No Change", which is a valid value. SimCity 4 just wipes out the reps entirely for "nulled" properties, it seems.

Fascinating. Concerning, as well. There are many prop packs on this site alone that override props from other prop packs, and add/remove various things. Unless there are more specific limits on what causes prop pox than what you've described, many people could be infected.

One example of many:

BSC Mega Props - JES Vol01. Instance 10A83059 (jes_BN-Line-53_Container) conflicts with the one contained in BSC MEGA Props - Misc Vol02. Same basic prop, different properties.

The BSC Mega Props - JES Vol01 version contains several more properties than the other. These are mostly binary in nature. MaxFireStage and Flammability are also not in the other version.

MEGA Props - Misc Vol02 has fewer properties, but has a family value; something that should never be empty. In addition it has an zero'd text key for the purposes of allowing the prop to properly appear as a ground model. Could this cause issue if this file is loaded first, then overridden by BSC Mega Props - JES Vol01?

If you upload your city somewhere, I can do testing for you following your methodology. I'll need to make my own prop lot, of course.

I wouldn't worry about the programming language, yet. Just having a functional tool will work.

[catty]

Hi Guys

Had read this post early in the day and have just been doing some update work on the plugins catalog at CB, when this plugin caught my eye

http://community.simtropolis.com/files/file/24165-time-of-day-property/

Had a quick look thru the Catalog and couldn't find any others that are designed to change the time of day.

-catty