• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Devaka Cooray
  • Liutauras Vilda
  • Jeanne Boyarsky
  • Bear Bibeault
Sheriffs:
  • Paul Clapham
  • Knute Snortum
  • Rob Spoor
Saloon Keepers:
  • Tim Moores
  • Ron McLeod
  • Piet Souris
  • Stephan van Hulst
  • Carey Brown
Bartenders:
  • Tim Holloway
  • Frits Walraven
  • Ganesh Patekar

random access encrypted data

 
Ranch Hand
Posts: 188
5
MS IE Notepad Suse
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
this is more a general question than language specific - so any welcome

lets assume this example: I distribut an application (could be anything like physics stuff, a game, what ever) and it ships with some main code and encrypted data files. The protection would be some like this: based on a-symetric crypto a valid license is checked against a server - if the server confirms the license it provides the key to decrypt the data files on the fly. I know this isn't secure, but lets assume a perfect world for this example. To save space I didnt put each data record in its own file but rather use archives to achieve a better compression ration.
For a real world example: GTA 4 and 5 do this, except the key is already present in the executeable and can be obtained without proofing a legitimate license to rockstar.
BTT: GTA uses an archieve format where each file is encrypted by its own with a TOC leading the data, wich is also encrypted. So to load a file, the engine loads the archive header, use it to load the TOC, then finds the location and length of compressed encrypted data and loads it.
My question: aside from this approach where each entry is encrypted by its own would it be possible to encrypt an archive as a whole and still maintain random access? Or does it have to be like GTA does it?

Matt
 
Saloon Keeper
Posts: 10434
223
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm going to assume that by "random access" you mean you can just move the file pointer to any position and start decrypting from there. You can only do that when using a block cipher, your data is aligned to the start of the block, and you use a poor block mode that doesn't retain state from previous blocks, like ECB. In short, you'd be better off not using encryption at all.

If your files are small enough, you could of course just encrypt the individual files and then store them in an archive normally. I don't see a need to encrypt the whole archive when the individual entries are already encrypted.

Actually, I don't see a need for encryption at all. It's been shown time and time again that DRM doesn't work and only hinders actual buyers. Anybody who goes on with such a scheme is a fool that purposely ignores evidence that has been around for years now.
 
Matt Wong
Ranch Hand
Posts: 188
5
MS IE Notepad Suse
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I total agree with you about DRM - we heared so much bad news. I'm part of the "real fans buy the games" group. Back in the early 2000 I ripped a few games, but scince I started working to earn my own money I bought each game.
I just asked if such would possible cause I'm interrested in comparing compression rates:
- does it matter if compression is used before/after encryption
- does it matter if each asset is compressed by its own or to just put all together and then compress the whole archive
The reason to think about such is when one have 10'000s of assets adding up to 10s of GB but to keep them easy loadable. Also there the structure would be relevant: what assests are always needed, wich are optional - is the loading behaviour predictable at time of creating the archives?
Sure, this is theoreticle as I will never be part of such a huge project. For my own current project: if you want to use it online with others you have to sign up for a client certificate wich is validate by server on connection (TLS has all I need - why not use it?). If someone does not good stuff the certificate is revoked by CRL/OCSP and connection is terminated. This way TLS reject at next connection during handshake client cert validation.

About encryption isn't useful: well, aside from reverse engineer the binary, the gta community got around by not posting the key itself, but a hash along the information, how long the key is and that it's inside the binary. So anyone can just use that hash and a moving window of the keys length to scan the binary -> encryption broken.
Also using different key for each user would be infeaseable - let alone the physical disc copies as one had to master each disk with different key - would cost much money. Even if one would re-key each batch it would not be worth it.
 
Stephan van Hulst
Saloon Keeper
Posts: 10434
223
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
So... You want to make a single 10GB archive?

...why?
 
Bartender
Posts: 20940
127
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I have paid extra to get non-DRM versions of programs. I am absolutely livid that I cannot download Nook Books to my public filesystems anymore - even when the books itself explicitly states that the author/publisher has specified in the body text of the book itself that the book be sold without DRM (which means that Barnes & Noble may actually be breaking the law and I've stopped buying ebooks from them). And I'm extremely glad I never bought any of the recently-extinct Microsoft DRM-ed books, because a refund doesn't give me use of the book I "owned", the replacement cost may now be higher, and in some cases, it may not be possible to replace the vanished book at all.

I have watched people crack dongles, and even been enlisted to brute-force a forgotten Excel password. You can count me firmly on the side that DRM is bad customer relations and a drain on resources better employed making a product that (most) people will pay for (at a reasonable price). Plus DRM just gives pirates something to do for amusement.

You will not find me in the aisles of Wal-Mart or any other store where the number of security personnel checking receipts is greater than the number of people actually accepting cash for my purchases. I will be honest. And you will treat me as innocent until proven guilty. Or I won't buy from you at all. I grew up before Reagan made everyone assume that every job applicant was a drug-addicted illegal alien until they provided explicit proof to the contrary and I'm rather cranky about that.

OK. Having said that, you can encrypt an entire archive (ZIP, TAR, or whatever), or you can encrypt all or some of the entries in the archive. The trade-off is that encrypting the entire archive is more secure and faster, but requires decrypting the entire directory at a minimum before you can access the individual entries. A lot of the finer points depend on the archive format. Tarballs, for example, are often compressed and/or encrypted as a unit, whereas ZIP files make a file-by-file determination as to the level (if any) of compression they use, and so can it be with encryption. Don't quote me, but I think each file in a tarball is directly preceded by its directory data but a ZIP puts the entire directory at the end of the file (with a pointer to it in the head). Other schemes are possible, and each works better or worse with specific encryption tactics.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!