SC4 Devotion Forum Archives

Sim City 4 Devotion Tools => Tools - General Discussion => Topic started by: memo on May 05, 2014, 01:19:08 PM

Title: JDatPacker
Post by: memo on May 05, 2014, 01:19:08 PM
Hi all,

I have written a very simple cross-platform dat-packing utility for the JVM. It is an alternative to wouanagaine's SC4DatPacker which is Windows-only and difficult to get to work in Wine (for me, at least). Because of that and because of the blatant issues in the Mac Aspyr port of the game (see here (http://community.simtropolis.com/topic/62708-game-is-unplayable-without-internet-connection/)), I decided to write a program of my own. It works for Windows, too.

JDatPacker is available on the LEX now. Download Version 0.1.3 here (http://sc4devotion.com/csxlex/lex_filedesc.php?lotGET=3140).

Source files (and ReadMe) at GitHub (https://github.com/memo33/JDatPacker).

Please test the program and report any issues in this topic.
Title: Re: JDatPacker: Testers needed
Post by: NCGAIO on May 05, 2014, 03:04:18 PM
 &apls
Title: Re: JDatPacker: Testers needed
Post by: vuWIN on May 10, 2014, 05:56:17 AM
Hi memo, thanks for the effort you put into this. Unfortunately, it seems the packer is not working on my mac (air 2013 + OS 10.9.2)

I got no error messages, yet it fails to produce any output in the target folder, no matter what.
Maybe I'm getting something wrong?

Thanks for the support
Title: Re: JDatPacker: Testers needed
Post by: memo on May 10, 2014, 07:44:13 AM
Thank you for testing and welcome to the forums. :)

Sorry for the trouble. The error reporting mechanism is not entirely complete, yet. Please start the program using the attached script; simply extract it to the same folder as the jar file and double-click the start script. This will open an additional terminal window in which the errors will be displayed. Please post the console output here.
Title: Re: JDatPacker: Testers needed
Post by: MayorTim on May 12, 2014, 10:36:23 AM
I've run into the same issue - no error message, just no output whatsoever.

I have these files:JdatPacker-0.1.0.jar, JDatPacker.properties, License, and Readme.md. So far haven't been able to find a way to open either the readme or the license, or use the properties file (double clicking tells me I don't have an application that can open the file.) So I'm not really sure how to get to the error terminal.
Title: Re: JDatPacker: Testers needed
Post by: memo on May 12, 2014, 11:50:03 AM
Thanks for taking the time to try the program. The readme, license and properties are ordinary text files that can be opened with any text editor, but you usually only need to open the readme file (which you can also view online here (https://github.com/memo33/JDatPacker)). It's probably due to the file extension that your system does not know a default program.

Anyway, please follow the instructions of my previous post – using the attached script file – to get the error output, so that I can examine the problem.
Title: Re: JDatPacker: Testers needed
Post by: MayorTim on May 12, 2014, 01:03:48 PM
Ah. Because they're Unix executables, even if they're just text, most people won't be able to open them without changing security preferences.

Upon changing my preferences, I opened the readme in Terminal (the only thing that would open it on my mac) and got this:

Last login: Mon May 12 15:55:35 on ttys000
Tims-MacBook-Pro:~ Tim$ /Users/Tim/Desktop/JDatPacker-0.1.0/README.md ; exit;
/Users/Tim/Desktop/JDatPacker-0.1.0/README.md: line 2: JDatPacker: command not found
/Users/Tim/Desktop/JDatPacker-0.1.0/README.md: line 3: ============: command not found
/Users/Tim/Desktop/JDatPacker-0.1.0/README.md: line 5: JDatPacker: command not found
/Users/Tim/Desktop/JDatPacker-0.1.0/README.md: line 6: plugin: command not found
/Users/Tim/Desktop/JDatPacker-0.1.0/README.md: line 7: syntax error near unexpected token `http://sc4devotion.com/csxlex/lex_filedesc.php?lotGET=26'
/Users/Tim/Desktop/JDatPacker-0.1.0/README.md: line 7: `[SC4DatPacker](http://sc4devotion.com/csxlex/lex_filedesc.php?lotGET=26)'
logout

[Process completed]


Double clicking the file you attached to your previous post, I get this:

Last login: Mon May 12 15:57:58 on ttys000
Tims-MacBook-Pro:~ Tim$ /Users/Tim/Desktop/JDatPacker-0.1.0/JDatPacker.command ; exit;
No Java runtime present, requesting install.
logout

[Process completed]



However, I just installed Java in order to use this, so I know it's up to date.

When I double click the command file with the JDatpacker.jar open (and supposedly completed) I get the error: "To use the "java," command-line tool you need to install a JDK."

I now have these files in one folder on my desktop:JDatPacker-0.1.0.jar, Jdatpacker.command, jdatpacker.properties, license, and readme.md.
Title: Re: JDatPacker: Testers needed
Post by: memo on May 13, 2014, 12:24:26 AM
Quote from: MayorTim on May 12, 2014, 01:03:48 PM
Ah. Because they're Unix executables, even if they're just text, most people won't be able to open them without changing security preferences.

Upon changing my preferences, I opened the readme in Terminal (the only thing that would open it on my mac) and got this:

Last login: Mon May 12 15:55:35 on ttys000
Tims-MacBook-Pro:~ Tim$ /Users/Tim/Desktop/JDatPacker-0.1.0/README.md ; exit;


Are you trying to execute the readme file? For just viewing its contents, try, if you are using the terminal, "less README.md", or if you are using the Finder, right-click > open with > TextEdit.

Also, except for the start script, none of the files has its executable bit set, on my end.

Quote from: MayorTim on May 12, 2014, 01:03:48 PM
Double clicking the file you attached to your previous post, I get this:

Last login: Mon May 12 15:57:58 on ttys000
Tims-MacBook-Pro:~ Tim$ /Users/Tim/Desktop/JDatPacker-0.1.0/JDatPacker.command ; exit;
No Java runtime present, requesting install.
logout

[Process completed]



However, I just installed Java in order to use this, so I know it's up to date.

When I double click the command file with the JDatpacker.jar open (and supposedly completed) I get the error: "To use the "java," command-line tool you need to install a JDK."

I now have these files in one folder on my desktop:JDatPacker-0.1.0.jar, Jdatpacker.command, jdatpacker.properties, license, and readme.md.

From your previous post, I gather that you did manage to start the application before, which means that you have an appropriate version of Java installed. There shouldn't be any need to install a JDK (for developers), only a JRE (for running Java applications) is required. The reason why you are not able to start the application using the start script probably is that the "java" command cannot be found. I have attached two more scripts in which I tried to guess your Java installation path. Let's see if that works.
Title: Re: JDatPacker: Testers needed
Post by: MayorTim on May 15, 2014, 10:39:47 AM
Jdatpacker2.command doesn't seem to be the correct directory (I think), as when I run it I get this:

Last login: Thu May 15 13:28:15 on ttys000
Tims-MacBook-Pro:~ Tim$ /Users/Tim/Desktop/JDatPacker-0.1.0/JDatPacker2.command ; exit;
/Users/Tim/Desktop/JDatPacker-0.1.0/JDatPacker2.command: line 3: /Library/Java/Home/bin/java: No such file or directory
logout

[Process completed]


The other file, jdatpacker3.command, gives me this:

Last login: Thu May 15 13:28:39 on ttys000
Tims-MacBook-Pro:~ Tim$ /Users/Tim/Desktop/JDatPacker-0.1.0/JDatPacker3.command ; exit;
No Java runtime present, requesting install.
logout

[Process completed]

As well as the pop-up saying that in order to use the Java command-line tool, I need to install a JDK.
Title: Re: JDatPacker: Testers needed
Post by: memo on May 16, 2014, 04:11:43 AM
Strange... the scripts should work without a JDK.

In any case, I have just updated the application to version 0.1.1 (see link in the first post). This will open a popup to display possible error messages, if any errors occur. Thus, the start scripts are not necessary; simply double-click the jar file again.
Title: Re: JDatPacker: Testers needed
Post by: MayorTim on May 16, 2014, 11:47:13 PM
I tried the new version but did not get an error popup, only one saying that datpacking was completed. It still didn't actually create anything though, unfortunately.
Title: Re: JDatPacker: Testers needed
Post by: memo on May 17, 2014, 10:10:46 AM
One more guess: Does your "source directory" not contain any subfolders? Only subfolders will be repackaged. If there are files in the root directory, those won't be touched. I once fell into that trap, too, but it is exactly the behaviour of SC4DatPacker.

Also, the subfolders (or subsubfolders, etc.) need to contain actual DBPF files, that is extensions "dat", "sc4model", "sc4desc", "sc4lot" (case-insensitive). Any other files will be ignored.

If this is not the case, I don't have any more ideas, as I can't replicate the problem. It would be nice to hear if it worked successfully for someone else.
Title: Re: JDatPacker: Testers needed
Post by: CasperVg on May 18, 2014, 03:00:27 AM
I just gave it a go, and .dat files were being generated succesfully. However, after datpacking was done the program crashed because it couldn't find/access the .properties file. JDatPacker was not running in a privileged directory as far as I know (I just unzipped it into my Downloads folder).

java.security.PrivilegedActionException: java.io.FileNotFoundException: JDatPacker.properties (Toegang geweigerd)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)
Caused by: java.io.FileNotFoundException: JDatPacker.properties (Toegang geweigerd)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(Unknown Source)
at java.io.FileOutputStream.<init>(Unknown Source)
at jdatpacker.Controller$$anonfun$main$1$$anonfun$apply$mcV$sp$1$$anon$1$$anonfun$done$1.apply(Controller.scala:73)
at jdatpacker.Controller$$anonfun$main$1$$anonfun$apply$mcV$sp$1$$anon$1$$anonfun$done$1.apply(Controller.scala:73)
at resource.DefaultManagedResource.open(AbstractManagedResource.scala:106)
at resource.AbstractManagedResource.acquireFor(AbstractManagedResource.scala:85)
at resource.ManagedResourceOperations$class.acquireAndGet(ManagedResourceOperations.scala:25)
at resource.AbstractManagedResource.acquireAndGet(AbstractManagedResource.scala:48)
at resource.ManagedResourceOperations$class.foreach(ManagedResourceOperations.scala:45)
at resource.AbstractManagedResource.foreach(AbstractManagedResource.scala:48)
at jdatpacker.Controller$$anonfun$main$1$$anonfun$apply$mcV$sp$1$$anon$1.done(Controller.scala:73)
at javax.swing.SwingWorker$5.run(Unknown Source)
at javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.run(Unknown Source)
at sun.swing.AccumulativeRunnable.run(Unknown Source)
at javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.actionPerformed(Unknown Source)
at javax.swing.Timer.fireActionPerformed(Unknown Source)
at javax.swing.Timer$DoPostEvent.run(Unknown Source)
at java.awt.event.InvocationEvent.dispatch(Unknown Source)
at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
at java.awt.EventQueue.access$400(Unknown Source)
at java.awt.EventQueue$3.run(Unknown Source)
at java.awt.EventQueue$3.run(Unknown Source)
... 9 more
Title: Re: JDatPacker: Testers needed
Post by: memo on May 20, 2014, 02:04:38 AM
Thanks Casper. I have updated the link in the first post to version 0.1.2. It will now display an error message in that event, but does not cause the program to freeze. I guess you'll need to check your directory permissions, if you want your preferences to be stored.
Title: Re: JDatPacker: Testers needed
Post by: CasperVg on May 20, 2014, 03:27:21 AM
There seems to be a different issue now. The program now shows the correct warning about missing permissions (running as Administrator fixes this and creates the .properties file correctly).

However, this exception keeps popping up every time now, both when running without administator rights (after closing the missing permissions warning) and with administrator rights (immediately after pressing start)

java.security.PrivilegedActionException: java.util.concurrent.ExecutionException: scdbpf.DbpfExceptions$DbpfIoException: Failed to rename temp file to destination file: E:\Mijn documenten\SimCity 4\Plugins_COTEST\LEX_TEST_000.dat
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)
Caused by: java.util.concurrent.ExecutionException: scdbpf.DbpfExceptions$DbpfIoException: Failed to rename temp file to destination file: E:\Mijn documenten\SimCity 4\Plugins_COTEST\LEX_TEST_000.dat
at java.util.concurrent.FutureTask.report(Unknown Source)
at java.util.concurrent.FutureTask.get(Unknown Source)
at javax.swing.SwingWorker.get(Unknown Source)
at jdatpacker.Controller$$anonfun$main$1$$anonfun$apply$mcV$sp$1$$anon$1.done(Controller.scala:83)
at javax.swing.SwingWorker$5.run(Unknown Source)
at javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.run(Unknown Source)
at sun.swing.AccumulativeRunnable.run(Unknown Source)
at javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.actionPerformed(Unknown Source)
at javax.swing.Timer.fireActionPerformed(Unknown Source)
at javax.swing.Timer$DoPostEvent.run(Unknown Source)
at java.awt.event.InvocationEvent.dispatch(Unknown Source)
at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
at java.awt.EventQueue.access$400(Unknown Source)
at java.awt.EventQueue$3.run(Unknown Source)
at java.awt.EventQueue$3.run(Unknown Source)
... 9 more
Caused by: scdbpf.DbpfExceptions$DbpfIoException: Failed to rename temp file to destination file: E:\Mijn documenten\SimCity 4\Plugins_COTEST\LEX_TEST_000.dat
at scdbpf.DbpfFile$$anonfun$write$1.apply(DbpfFile.scala:222)
at scdbpf.DbpfFile$$anonfun$write$1.apply(DbpfFile.scala:211)
at rapture.core.strategy$ThrowExceptions.wrap(strategy.scala:52)
at scdbpf.DbpfFile$.write(DbpfFile.scala:211)
at jdatpacker.Model$.jdatpacker$Model$$writeEntries(Model.scala:68)
at jdatpacker.Model$$anonfun$pack$2.apply(Model.scala:79)
at jdatpacker.Model$$anonfun$pack$2.apply(Model.scala:76)
at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:778)
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:777)
at jdatpacker.Model$.pack(Model.scala:76)
at jdatpacker.Controller$$anonfun$main$1$$anonfun$apply$mcV$sp$1$$anon$1.doInBackground(Controller.scala:66)
at jdatpacker.Controller$$anonfun$main$1$$anonfun$apply$mcV$sp$1$$anon$1.doInBackground(Controller.scala:65)
at javax.swing.SwingWorker$1.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at javax.swing.SwingWorker.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)


After that the programming hangs until I force close it.
Title: Re: JDatPacker: Testers needed
Post by: memo on May 20, 2014, 07:04:44 AM
Thanks for the quick response! I can now replicate this problem in Windows. The problem is that the program is hindered from overwriting existing dat files, which I'll have to fix in my DBPF library. For the time being, if you are experiencing this problem, you can avoid this by deleting preexisting dat files from the target folder.
Title: Re: JDatPacker: Testers needed
Post by: memo on May 23, 2014, 04:29:20 AM
The link in the first post has been updated to version 0.1.3, which fixes the aforementioned (Windows-only) problem.
Title: JDatPacker: available on LEX now
Post by: memo on November 14, 2014, 03:32:26 PM
JDatPacker is available on the LEX now (see first post for link). It is still the same version as before (v0.1.3) – no changes have been made.
Title: Re: JDatPacker
Post by: danwaxx on December 15, 2014, 05:36:08 PM
It is failing on my computer! I am now packing my plugins folder but still getting brown boxes. Any ideas for a workaround? :angrymore:
Title: Re: JDatPacker
Post by: memo on December 16, 2014, 12:03:48 AM
Do I understand you correctly, the packaging process of your plugins works successfully and your game doesn't crash, but you get brown boxes? This sounds more like you are missing a dependency. Could you show a picture, please?
Title: Re: JDatPacker
Post by: daddy32992 on August 09, 2015, 10:15:21 AM
Hey all. I'm having trouble using JdatPacker. Im very new to this so I really don't have a clue where I'm going wrong. Im using a mac version of SC4. My main issue is that my game crashes (just a black screen) upon opening, so I downloaded Jdatpacker to use on the plugin folder (I'm trying to get the NamNetworkaddon to work) but now I'm stuck. Can someone help please?
Title: Re: JDatPacker
Post by: memo on August 10, 2015, 12:55:29 AM
I'm afraid you need to provide a bit more information on what you get stuck with, as it is impossible to help you, otherwise.
Title: Re: JDatPacker
Post by: nirasa on September 30, 2016, 02:12:40 AM
Thank you. Nice program. It works very well in LinuxMint 18.
Title: Re: JDatPacker
Post by: ChiefZDN on April 23, 2017, 03:30:01 AM
Why I can't datpack the DAT correctly? I'm want to merge SimCity_1.dat with the CAM I-HT patch file. I'm using OpenJDK 8 (JDK, not JRE) on elementary OS 0.4 Loki (based on Ubuntu 16.04). The packer is running like normal, but the processing is fast enough, although I'm using a mid-end laptop (Core i3). And, when I check the destination folder, I saw the folder is still empty. I was debugging with the terminal and the program sent (100,).
Title: Re: JDatPacker
Post by: mgb204 on April 23, 2017, 03:10:33 PM
Have you tried installing the JRE and using that, since the JDK isn't really meant for running Java applications so much as making/testing them? It could be what's causing your problem. To tell you what the code 100 means, you'd need to speak to the developer, Memo, who hasn't been around for a long time now. So that's really not any help with determining the issue.
Title: Re: JDatPacker
Post by: ChiefZDN on April 23, 2017, 05:24:11 PM
Quote from: mgb204 on April 23, 2017, 03:10:33 PM
Have you tried installing the JRE and using that, since the JDK isn't really meant for running Java applications so much as making/testing them? It could be what's causing your problem. To tell you what the code 100 means, you'd need to speak to the developer, Memo, who hasn't been around for a long time now. So that's really not any help with determining the issue.

OpenJDK 8 (JDK edition) already include the JRE. The installation is completed without any errors.
Title: Re: JDatPacker
Post by: mgb204 on April 24, 2017, 05:59:00 AM
The point is that you need to run the app through the JRE, not the JDK. It probably doesn't matter which install the JRE came from, then again they may not be identical, so it might be worth a try?

Datpacking is a straightforward enough process, if it's not outputting a packed file, I can only assume there is an issue somewhere with compatibility and your system. Because a JAVA app isn't installed code, it's just run from it's container, that really shouldn't be a problem. Of course if changes in Java have led to an issue with the code, you may find an older version of Java works? But frankly I'm just speculating here.

I guess the first thing to check is whether it can dat pack any files at all, or if the issue is specific to SimCity_1.dat. Given that's a really large file, it takes more resources to handle the operation. After that make sure you have full permissions for running the app and accessing the input/output folders. From the command line you may need to use SUDO to elevate the permissions given to the app. Otherwise be sure you are checking the correct output location for the dat packed files.
Title: Re: JDatPacker
Post by: eagle74 on June 12, 2017, 07:12:13 AM
Running Jdat Packer to pack SimCity_1 with the IHT fix dat file does not give any output to the destination folder.  The program runs for several seconds & says it completed successfully, but the output folder is still empty.  I put the two files to be packed in a subfolder of the source directory on my desktop & created a new folder on my desktop for the output.  I am running Win 7 64 bit with the latest version of Java.
Title: Re: JDatPacker
Post by: etg-nc2nyc on April 16, 2018, 10:12:56 AM
I realize this is an old thread, but I'll take my chances.  :)

I'm running OSX El Capitan 10.11.6 on a macbook pro. My Java version is Java 8 Update 161.

When I run/open JDatPacker-0.1.3.jar via Jar Launcher 15.0.1 (the only option I see and it's the default if I just double click the .jar file), I get the following error:
"The Java JAR file 'JDatPacker-0.1.3.jar' could not be launched. Check the Console for possible error messages."

I'm not sure what or where the "Console" is and if I did, I wouldn't understand it, but if you need it to help me, I will seek it out and post it.

Can you help?
Thank you!
Title: Re: JDatPacker
Post by: etg-nc2nyc on April 16, 2018, 04:56:28 PM
Just popping back by to say on a wild chance I thought I would download from Github and try... that worked! ;D Not sure why. Anyway... great product. Thank you for the hard work!  &apls :thumbsup: