• Welcome to SC4 Devotion Forum Archives.

Mega Zips (.EXE-ORCISM)

Started by Handyman, May 14, 2020, 05:46:05 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Handyman

To get this topic started I would like to explain why.

This project was started when xdita began to release his Mega Zips on the Lex. I had extra time because of the Covid19 shutdown and was looking for another project to further my java knowledge. (Full disclosure: I am not a professional programmer, this is a hobby for me).

My first goal was to expand all the zip files into a folder structure that was consistent and easy to use. This was harder than first thought because of all the different ways files are presented within the zips.This part is now complete and the first two pictures below demonstrate this.
The first picture shows the zip structure.
The second picture shows the resulting file structure.
So this part works well now and I also use this process to create a list of all the ".exe Installers' in the Mega Zip.

During this time I read many posts regarding the general dislike for the '.exe Installers' So I went on to find out what work had gone into a solution and if there was some way to do it programmatically. I found nothing I could use but came across information about a java robot method. This led to updating a process I used many years ago.

The ultimate goal of this project is an automated way to process the Mega Zip into a new Mega Zip with organized folders and zero '.exe Installers with the hope this could replace the original Mega Zips for downloading.

My personal opinion is that some '.exe installers', like NAM and CAM, should remain as is and maybe select others as well.

I continue to find small differences in the panels the installers present which can be a problem for the robot. I can deal with the DN/MN panels and the Create Folder dialog but I will have to develop a way to recognize differences like when there is no panel that asks to agree to terms and conditions.

Also I currently have no way to know (in advance) which installers present a choice for MN/DN. If any one knows of a list I would be grateful.

xxdita

Thanks for posting this over here. And thank you for adding something productive to the conversation.

To start, I'll paraphrase what I've said in the Shared Plugins Folders thread:
Installers exist outside of the LEX and BSC are not the only ones to have used them. So any program/app needs to be able to handle such things on a user level, the way File Juicer does for MAC users. I continue looking for a Windows or even Android program that can do this, but have had no luck so far.

Unfortunately, I don't see another way to upload the BSC LEX Superior Collections that is in line with our own LEX rules and long-established community standards. I never would have put these first 3 Collections together at all had it not been for the LEX's Dependency Tracker feature.

These Collections are the largest curated sets of custom content available on any SC4 exchange that aren't behind a paywall. Unlike physical or digital copies of DVD's, I am able to provide real time tech support in generally a matter of minutes after seeing a problematic file reported. I would not be able to do that with a 500MB upload. Nor would I be able to add or remove files on the fly as I can now.

I do understand the inconvenience of running so many installers at once. I've done it probably a couple dozen times myself. But I also know that each file in these Collections has been tested and verified by some of the best in the game, and I am willing to provide tech support for any minor errors that may be found.

So if it's possible, I would like you to focus on finding a way to use what you have so far to be able to handle whatever files someone may download. The way the Dependency Tracker works, a Mega Zip can be different for each person that downloads it, depending on what files they've already downloaded and hopefully installed.

Whether it's a single lot with just a couple dependencies, or one of my Collections with hundreds of files, I think you're on to something.

Handyman

Quote from: xxdita on May 14, 2020, 07:18:23 AM
The way the Dependency Tracker works, a Mega Zip can be different for each person that downloads it, depending on what files they've already downloaded and hopefully installed.

Whether it's a single lot with just a couple dependencies, or one of my Collections with hundreds of files, I think you're on to something.

At this point the robot will find whatever installers are in the download (once it is expanded). There are some issues as mentioned above and I still have to find a way to maintain focus on the installer after the robot starts. I will keep looking and if any members have some suggestions they will be welcome.

I cannot seem to get my robot video clip down below 2000kb for uploading here so I have added a link to it on Simtropolis.

https://community.simtropolis.com/applications/core/interface/file/attachment.php?id=57967

xxdita

Quote from: Handyman on May 14, 2020, 09:44:24 AM
Quote from: xxdita on May 14, 2020, 07:18:23 AM
The way the Dependency Tracker works, a Mega Zip can be different for each person that downloads it, depending on what files they've already downloaded and hopefully installed.

Whether it's a single lot with just a couple dependencies, or one of my Collections with hundreds of files, I think you're on to something.

At this point the robot will find whatever installers are in the download (once it is expanded). There are some issues as mentioned above and I still have to find a way to maintain focus on the installer after the robot starts. I will keep looking and if any members have some suggestions they will be welcome.

I cannot seem to get my robot video clip down below 2000kb for uploading here so I have added a link to it on Simtropolis.

https://community.simtropolis.com/applications/core/interface/file/attachment.php?id=57967


That looks like a damn near perfect solution to me. Or as close as it's going to get. It seems to be everything I had hoped for.

One of the concerns raised about installers is (false) virus warnings. Have you run into any of these? Can your bot give a prompt or a log to report them if/when they occur?

Handyman

Virus software problems are something I do not encounter on my Linux system  :bnn:  and I have no windows systems available to me. If I had known I was going to be working on this I would have left windows on my new laptop and dual booted it but I wiped it out 30 minutes after the laptop arrived.  At some point I will need to put together a scaled down test for someone to run.

xxdita

Quote from: Handyman on May 15, 2020, 04:17:57 AM
Virus software problems are something I do not encounter on my Linux system  :bnn:  and I have no windows systems available to me. If I had known I was going to be working on this I would have left windows on my new laptop and dual booted it but I wiped it out 30 minutes after the laptop arrived.  At some point I will need to put together a scaled down test for someone to run.

How long does it take the bot to process one of the Collections fully?

Handyman

Quote from: xxdita on May 15, 2020, 04:22:04 AM
How long does it take the bot to process one of the Collections fully?

I have only run a full test on the first Collection and it took about 15 minutes.

Handyman

Update; I am currently researching another idea to maintain control of the robot which involves monitoring system processes (PID's). I have used a similar approach in a bash project but this time it will also have to work on a windows machine so there is much research to be done.

Just to clarify why I am using a robot if it is so hard to control. At the current time there is no way to just extract the files from the installers without running them which can be very tedious if you have a lot of them. So after a lot of time spent on dead end roads I landed on the java robot class which I can use to send virtual keystrokes to the hot keys in the installer. A major caveat to this process is that the robot does not own the installer which means the installer must retain the focus in order to receive the keystrokes. If another app or process steals said focus they now also receive the keystrokes. As you can imagine this could lead to unwanted outcomes.
So my current direction is to try and detect if another app steals the focus and pause the robot long enough to steal it back and continue.

Any thoughts or input s are welcome.

Handyman

Unfortunately I must announce that the release of the robot is not going to be an option. I cannot, in good conscience, release something that could do harm to someone, digital or otherwise. At this time I do not have the ability to secure it unless it is used specifically to control java components. So it will remain in its cage for now.

I continue to believe there can be a solution that works for everybody and I am trying to remained focused in that direction. We learn from our failures and I have learned a lot.

This happens to be a very interesting time in the world that demands a lot of our attention so progress on this will obviously slow down for some time to come.
Trust in the future.