35 bool kill_cmd =
false;
37 bool compress_flag =
false;
38 int default_compression_level = 6;
40 bool got_multifile_name =
false;
41 bool to_stdout =
false;
42 bool encryption_flag =
false;
44 bool got_password =
false;
46 bool got_header_prefix =
false;
48 bool got_chdir_to =
false;
49 size_t scale_factor = 0;
52 vector_string sign_params;
55 string dont_compress_str =
"jpg,png,mp3,ogg";
58 string text_ext_str =
"txt";
60 bool got_record_timestamp_flag =
false;
61 bool record_timestamp_flag =
true;
74 const char *nptr = str.c_str();
76 int result = strtol(nptr, &endptr, 10);
96 "Usage: multify -[c|r|u|t|x] -f <multifile_name> [options] <subfile_name> ...\n";
103 "multify is used to store and extract files from a Panda Multifile.\n" 104 "This is similar to a tar or zip file in that it is an archive file that\n" 105 "contains a number of subfiles that may later be extracted.\n\n" 107 "Panda's VirtualFileSystem is capable of mounting Multifiles for direct\n" 108 "access to the subfiles contained within without having to extract them\n" 109 "out to independent files first.\n\n" 111 "The command-line options for multify are designed to be similar to those\n" 112 "for tar, the traditional Unix archiver utility.\n\n" 116 " You must specify exactly one of the following command switches:\n\n" 119 " Create a new Multifile archive. Subfiles named on the command line\n" 120 " will be added to the new Multifile. If the Multifile already exists,\n" 121 " it is first removed.\n\n" 124 " Rewrite an existing Multifile archive. Subfiles named on the command\n" 125 " line will be added to the Multifile or will replace subfiles within\n" 126 " the Multifile with the same name. The Multifile will be repacked\n" 127 " after completion, even if no Subfiles were added.\n\n" 130 " Update an existing Multifile archive. This is similar to -r, except\n" 131 " that files are compared byte-for-byte with their corresponding files\n" 132 " in the archive first. If they have not changed, the multifile is not\n" 133 " modified (other than to repack it if necessary).\n\n" 136 " List the contents of an existing Multifile. With -v, this shows\n" 137 " the size of each Subfile and its compression ratio, if compressed.\n\n" 140 " Extract the contents of an existing Multifile. The Subfiles named on\n" 141 " the command line, or all Subfiles if nothing is named, are extracted\n" 142 " into the current directory or into whichever directory is specified\n" 146 " Delete (kill) the named Subfiles from the Multifile. The Multifile\n" 147 " will be repacked after completion.\n\n" 150 " You must always specify the following switch:\n\n" 152 " -f <multifile_name>\n" 153 " Names the Multifile that will be operated on.\n\n\n" 155 " The remaining switches are optional:\n\n" 158 " Run verbosely. In -c, -r, or -x mode, list each file as it is\n" 159 " written or extracted. In -t mode, list more information about each\n" 163 " Compress subfiles as they are written to the Multifile. Unlike tar\n" 164 " (but like zip), subfiles are compressed individually, instead of the\n" 165 " entire archive being compressed as one stream. It is not necessary\n" 166 " to specify -z when extracting compressed subfiles; they will always be\n" 167 " decompressed automatically. Also see -Z, which restricts which\n" 168 " subfiles will be compressed based on the filename extension.\n\n" 171 " Encrypt subfiles as they are written to the Multifile using the password\n" 172 " specified with -p, below. Subfiles are encrypted individually, rather\n" 173 " than encrypting the entire multifile, and different subfiles may be\n" 174 " encrypted using different passwords (although this requires running\n" 175 " multify multiple times). It is not possible to encrypt the multifile's\n" 176 " table of contents using this interface, but see the pencrypt program to\n" 177 " encrypt the entire multifile after it has been generated.\n\n" 181 " Specifies the password to encrypt or decrypt subfiles. If this is not\n" 182 " specified, and passwords are required, the user will be prompted from\n" 183 " standard input.\n\n" 186 " Specifies a header_prefix to write to the beginning of the multifile.\n" 187 " This is primarily useful for creating a multifile that can be invoked\n" 188 " directly as a program from the shell on Unix-like environments,\n" 189 " for instance, p3d files. The header_prefix must begin with a hash\n" 190 " mark and end with a newline; this will be enforced if it is not\n" 191 " already so. This only has effect in conjunction with with -c, -u,\n" 194 " -F <scale_factor>\n" 195 " Specify a Multifile scale factor. This is only necessary to support\n" 196 " Multifiles that will exceed 4GB in size. The default scale factor is\n" 197 " 1, which should be sufficient for almost any application, but the total\n" 198 " size of the Multifile will be limited to 4GB * scale_factor. The size\n" 199 " of individual subfiles may not exceed 4GB in any case.\n\n" 201 " -C <extract_dir>\n" 203 " Change to the named directory before working on files;\n" 204 " that is, extraction/creation/update and replace will be based on this path\n\n" 207 " With -x, extract subfiles to standard output instead of to disk.\n\n" 208 " -Z <extension_list>\n" 209 " Specify a comma-separated list of filename extensions that represent\n" 210 " files that are not to be compressed. The default if this is omitted is\n" 211 " \"" << dont_compress_str <<
"\". Specify -Z \"\" (be sure to include the space) to allow\n" 212 " all files to be compressed.\n\n" 213 " -X <extension_list>\n" 214 " Specify a comma-separated list of filename extensions that represent\n" 215 " text files. These files are opened and read in text mode, and added to\n" 216 " the multifile with the text flag set. The default if this is omitted is\n" 217 " \"" << text_ext_str <<
"\". Specify -X \"\" (be sure to include the space) to record\n" 218 " all files in binary mode.\n\n" 221 " Enable or disable the recording of file timestamps within the multifile.\n" 222 " If <flag> is 1, timestamps will be recorded within the multifile for\n" 223 " each subfile added; this is the default behavior. If <flag> is 0,\n" 224 " timestamps will not be recorded, which will make it easier to do a\n" 225 " bitwise comparison between multifiles to determine whether their\n" 226 " contents are equivalent.\n\n" 229 " Specify the compression level when -z is in effect. Larger numbers\n" 230 " generate slightly smaller files, but compression takes longer. The\n" 231 " default is -" << default_compression_level <<
".\n\n" 233 " -S file.crt[,chain.crt[,file.key[,\"password\"]]]\n" 234 " Sign the multifile. The signing certificate should be in PEM form in\n" 235 " file.crt, with its private key in PEM form in file.key. If the key\n" 236 " is encrypted on-disk, specify the decryption password as the third\n" 237 " option. If a certificate chain is required, chain.crt should also\n" 238 " be specified; note that the separating commas should be supplied\n" 239 " even if this optional filename is omitted.\n" 240 " You may also provide a single composite file that contains the\n" 241 " certificate, chain, and key in the same file.\n" 242 " PEM form is the form accepted by the Apache web server. The\n" 243 " signature is written to the multifile to prove it is unchanged; any\n" 244 " subsequent change to the multifile will invalidate the signature.\n" 245 " This parameter may be repeated to sign the multifile with additional\n" 246 " certificates.\n\n";
252 cerr <<
"Enter password: ";
253 std::getline(std::cin, password);
262 is_named(
const string &subfile_name,
const vector_string ¶ms) {
265 if (params.empty()) {
270 vector_string::const_iterator pi;
271 for (pi = params.begin(); pi != params.end(); ++pi) {
272 if (subfile_name == (*pi)) {
281 is_text(
const Filename &subfile_name) {
286 if (text_ext.find(ext) != text_ext.end()) {
295 get_compression_level(
const Filename &subfile_name) {
297 if (!compress_flag) {
303 if (dont_compress.find(ext) != dont_compress.end()) {
309 return default_compression_level;
319 cerr <<
"Unable to scan directory " << directory_name <<
"\n";
324 filenames.reserve(files.size());
325 vector_string::const_iterator fi;
326 for (fi = files.begin(); fi != files.end(); ++fi) {
327 Filename subfile_name(directory_name, (*fi));
328 filenames.push_back(subfile_name);
331 return do_add_files(multifile, filenames);
338 for (fi = filenames.begin(); fi != filenames.end(); ++fi) {
342 if (!do_add_directory(multifile, subfile_name)) {
346 }
else if (!subfile_name.
exists()) {
347 cerr <<
"Not found: " << subfile_name <<
"\n";
351 if (is_text(subfile_name)) {
357 string new_subfile_name;
360 (subfile_name, subfile_name, get_compression_level(subfile_name));
363 (subfile_name, subfile_name, get_compression_level(subfile_name));
365 if (new_subfile_name.empty()) {
366 cerr <<
"Unable to add " << subfile_name <<
".\n";
370 cout << new_subfile_name <<
"\n";
379 add_files(
const vector_string ¶ms) {
381 if (append || update) {
383 cerr <<
"Unable to open " << multifile_name <<
" for updating.\n";
388 cerr <<
"Unable to open " << multifile_name <<
" for writing.\n";
393 if (got_record_timestamp_flag) {
397 if (encryption_flag) {
402 if (got_header_prefix) {
407 cerr <<
"Setting scale factor to " << scale_factor <<
"\n";
412 filenames.reserve(params.size());
413 vector_string::const_iterator si;
414 for (si = params.begin(); si != params.end(); ++si) {
416 filenames.push_back(subfile_name);
420 if (got_chdir_to && !chdir_to.
chdir()) {
421 cout <<
"Failed to chdir to " << chdir_to <<
": " << strerror(errno) << endl;
425 bool okflag = do_add_files(multifile, filenames);
434 if (!multifile->
repack()) {
435 cerr <<
"Failed to write " << multifile_name <<
".\n";
439 if (!multifile->
flush()) {
440 cerr <<
"Failed to write " << multifile_name <<
".\n";
449 extract_files(
const vector_string ¶ms) {
450 if (!multifile_name.
exists()) {
451 cerr << multifile_name <<
" not found.\n";
455 if (!multifile->open_read(multifile_name)) {
456 cerr <<
"Unable to open " << multifile_name <<
" for reading.\n";
466 bool any_encrypted =
false;
467 for (i = 0; i < num_subfiles && !any_encrypted; i++) {
469 if (is_named(subfile_name, params)) {
471 any_encrypted =
true;
481 for (i = 0; i < num_subfiles; i++) {
483 if (is_named(subfile_name, params)) {
486 filename =
Filename(chdir_to, subfile_name);
490 cerr << filename <<
"\n";
495 cout << filename <<
"\n";
506 kill_files(
const vector_string ¶ms) {
507 if (!multifile_name.
exists()) {
508 cerr << multifile_name <<
" not found.\n";
513 cerr <<
"Unable to open " << multifile_name <<
" for read/write.\n";
517 if (got_header_prefix) {
522 while (i < multifile->get_num_subfiles()) {
524 if (is_named(subfile_name, params)) {
528 cout << filename <<
"\n";
539 if (!multifile->
repack()) {
540 cerr <<
"Failed to write " << multifile_name <<
".\n";
544 if (!multifile->
flush()) {
545 cerr <<
"Failed to write " << multifile_name <<
".\n";
556 cerr <<
"Cannot sign multifiles without OpenSSL compiled in.\n";
559 #else // HAVE_OPENSSL 564 cerr <<
"Unable to re-open " << multifile_name <<
" for signing.\n";
568 vector_string::iterator si;
569 for (si = sign_params.begin(); si != sign_params.end(); ++si) {
570 const string ¶m = (*si);
574 size_t comma1 = param.find(
',');
575 if (comma1 == string::npos) {
579 size_t comma2 = param.find(
',', comma1 + 1);
580 if (comma2 == string::npos) {
584 size_t comma3 = param.find(
',', comma2 + 1);
585 if (comma3 == string::npos) {
589 password = param.substr(comma3 + 1);
594 if (!multifile->add_signature(certificate, chain, pkey, password)) {
600 #endif // HAVE_OPENSSL 604 format_timestamp(
bool record_timestamp, time_t timestamp) {
605 static const size_t buffer_size = 512;
606 static char buffer[buffer_size];
608 if (!record_timestamp) {
613 if (timestamp == 0) {
615 return " (no date) ";
618 time_t now = time(
nullptr);
619 struct tm *tm_p = localtime(×tamp);
621 if (timestamp > now || (now - timestamp > 86400 * 365)) {
624 strftime(buffer, buffer_size,
"%b %d %Y", tm_p);
627 strftime(buffer, buffer_size,
"%b %d %H:%M", tm_p);
634 list_files(
const vector_string ¶ms) {
635 if (!multifile_name.
exists()) {
636 cerr << multifile_name <<
" not found.\n";
643 if (istr ==
nullptr) {
644 cerr <<
"Unable to open " << multifile_name <<
" for reading.\n";
649 if (!multifile->open_read(
new IStreamWrapper(istr,
true),
true)) {
650 cerr <<
"Unable to open " << multifile_name <<
" for reading.\n";
658 cout << num_subfiles <<
" subfiles:\n" << std::flush;
659 for (i = 0; i < num_subfiles; i++) {
661 if (is_named(subfile_name, params)) {
662 char encrypted_symbol =
' ';
664 encrypted_symbol =
'e';
666 char text_symbol =
' ';
674 if (orig_length != 0) {
675 ratio = (double)internal_length / (
double)orig_length;
678 printf(
"%12d worse %c%c %s %s\n",
680 encrypted_symbol, text_symbol,
683 subfile_name.c_str());
685 printf(
"%12d %3.0f%% %c%c %s %s\n",
687 100.0 - ratio * 100.0,
688 encrypted_symbol, text_symbol,
691 subfile_name.c_str());
694 printf(
"%12d %c%c %s %s\n",
696 encrypted_symbol, text_symbol,
699 subfile_name.c_str());
706 cout <<
"Last modification " 707 << format_timestamp(
true, multifile->
get_timestamp()) <<
"\n";
714 cout <<
"Multifile needs to be repacked.\n";
717 for (i = 0; i < num_subfiles; i++) {
719 if (is_named(subfile_name, params)) {
720 cout << subfile_name <<
"\n";
726 int num_signatures = multifile->get_num_signatures();
727 if (num_signatures != 0) {
729 for (i = 0; i < num_signatures; ++i) {
730 cout <<
"Signed by " << multifile->get_signature_friendly_name(i);
731 int verify_result = multifile->validate_signature_certificate(i);
732 if (verify_result == 0) {
733 cout <<
" (certificate validated)\n";
735 cout <<
" (certificate unknown, reason " << verify_result <<
")\n";
738 multifile->write_signature_certificate(i, cout);
743 #endif // HAVE_OPENSSL 749 tokenize_extensions(
const string &str,
pset<string> &extensions) {
751 while (p < str.length()) {
752 size_t q = str.find_first_of(
",", p);
753 if (q == string::npos) {
754 extensions.insert(str.substr(p));
757 extensions.insert(str.substr(p, q - p));
760 extensions.insert(
string());
764 main(
int argc,
char **argv) {
774 if (*argv[1] !=
'-' && *argv[1] !=
'\0') {
775 char *new_arg = (
char *)PANDA_MALLOC_ARRAY(strlen(argv[1]) + 2);
777 strcpy(new_arg + 1, argv[1]);
784 static const char *optflags =
"crutxkvz123456789Z:T:X:S:f:OC:ep:P:F:h";
785 int flag = getopt(argc, argv, optflags);
787 while (flag != EOF) {
811 compress_flag =
true;
814 default_compression_level = 1;
815 compress_flag =
true;
818 default_compression_level = 2;
819 compress_flag =
true;
822 default_compression_level = 3;
823 compress_flag =
true;
826 default_compression_level = 4;
827 compress_flag =
true;
830 default_compression_level = 5;
831 compress_flag =
true;
834 default_compression_level = 6;
835 compress_flag =
true;
838 default_compression_level = 7;
839 compress_flag =
true;
842 default_compression_level = 8;
843 compress_flag =
true;
846 default_compression_level = 9;
847 compress_flag =
true;
850 dont_compress_str = optarg;
853 text_ext_str = optarg;
856 sign_params.push_back(optarg);
862 (flag != 0 && flag != 1)) {
863 cerr <<
"Invalid timestamp flag: " << optarg <<
"\n";
867 record_timestamp_flag = (flag != 0);
868 got_record_timestamp_flag =
true;
873 got_multifile_name =
true;
883 encryption_flag =
true;
890 header_prefix = optarg;
891 got_header_prefix =
true;
896 scale_factor = strtol(optarg, &endptr, 10);
897 if (*endptr !=
'\0') {
898 cerr <<
"Invalid integer: " << optarg <<
"\n";
902 if (scale_factor == 0) {
903 cerr <<
"Scale factor must be nonzero.\n";
917 cerr <<
"Unhandled switch: " << flag << endl;
920 flag = getopt(argc, argv, optflags);
922 argc -= (optind - 1);
923 argv += (optind - 1);
926 if ((create?1:0) + (append?1:0) + (update?1:0) + (tlist?1:0) + (extract?1:0) + (kill_cmd?1:0) != 1) {
927 cerr <<
"Exactly one of -c, -r, -u, -t, -x, -k must be specified.\n";
932 if (!got_multifile_name) {
933 cerr <<
"Multifile name not specified.\n";
939 tokenize_extensions(dont_compress_str, dont_compress);
942 tokenize_extensions(text_ext_str, text_ext);
945 vector_string params;
946 params.reserve(argc - 1);
947 for (
int i = 1; i < argc; i++) {
948 params.push_back(argv[i]);
952 if (create || append || update) {
953 okflag = add_files(params);
954 }
else if (extract) {
955 if (got_record_timestamp_flag) {
956 cerr <<
"Warning: -T ignored on extract.\n";
958 okflag = extract_files(params);
959 }
else if (kill_cmd) {
960 if (got_record_timestamp_flag) {
961 cerr <<
"Warning: -T ignored on kill.\n";
963 okflag = kill_files(params);
965 if (got_record_timestamp_flag) {
966 cerr <<
"Warning: -T ignored on list.\n";
968 okflag = list_files(params);
971 if (okflag && !sign_params.empty()) {
int string_to_int(const string &str, string &tail)
A string-interface wrapper around the C library strtol().
size_t get_scale_factor() const
Returns the internal scale factor for this Multifile.
size_t get_subfile_internal_length(int index) const
Returns the number of bytes the indicated subfile consumes within the archive.
bool open_write(const Filename &multifile_name)
Opens the named Multifile on disk for writing.
time_t get_timestamp() const
Returns the modification timestamp of the overall Multifile.
A hierarchy of directories and files that appears to be one continuous file system,...
bool needs_repack() const
Returns true if the Multifile index is suboptimal and should be repacked.
std::istream * open_read_file(const Filename &filename, bool auto_unwrap) const
Convenience function; returns a newly allocated istream if the file exists and can be read,...
void set_binary()
Indicates that the filename represents a binary file.
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
void set_text()
Indicates that the filename represents a text file.
bool is_subfile_compressed(int index) const
Returns true if the indicated subfile has been compressed when stored within the archive,...
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
void set_encryption_flag(bool flag)
Sets the flag indicating whether subsequently-added subfiles should be encrypted before writing them ...
bool is_subfile_encrypted(int index) const
Returns true if the indicated subfile has been encrypted when stored within the archive,...
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
bool chdir() const
Changes directory to the specified location.
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
bool repack()
Forces a complete rewrite of the Multifile and all of its contents, so that its index will appear at ...
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
size_t get_subfile_length(int index) const
Returns the uncompressed data length of the nth subfile.
The name of a file, such as a texture file or an Egg file.
get_subfile_name
Returns the name of the nth subfile.
std::string update_subfile(const std::string &subfile_name, const Filename &filename, int compression_level)
Adds a file on disk to the subfile.
This class provides a locking wrapper around an arbitrary istream pointer.
time_t get_subfile_timestamp(int index) const
Returns the modification time of the nth subfile.
void set_scale_factor(size_t scale_factor)
Changes the internal scale factor for this Multifile.
bool open_read_write(const Filename &multifile_name)
Opens the named Multifile on disk for reading and writing.
static VirtualFileSystem * get_global_ptr()
Returns the default global VirtualFileSystem.
bool extract_subfile_to(int index, std::ostream &out)
Extracts the nth subfile to the indicated ostream.
bool flush()
Writes all contents of the Multifile to disk.
bool get_record_timestamp() const
Returns the flag indicating whether timestamps should be recorded within the Multifile or not.
void set_encryption_password(const std::string &encryption_password)
Specifies the password that will be used to encrypt subfiles subsequently added to the multifile,...
bool extract_subfile(int index, const Filename &filename)
Extracts the nth subfile into a file with the given name.
void set_record_timestamp(bool record_timestamp)
Sets the flag indicating whether timestamps should be recorded within the Multifile or not.
void remove_subfile(int index)
Removes the nth subfile from the Multifile.
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
std::string get_extension() const
Returns the file extension.
void preprocess_argv(int &argc, char **&argv)
Processes the argc, argv pair as needed before passing it to getopt().
get_num_subfiles
Returns the number of subfiles within the Multifile.
bool is_directory() const
Returns true if the filename exists and is a directory name, false otherwise.
A file that contains a set of files.
void set_header_prefix(const std::string &header_prefix)
Sets the string which is written to the Multifile before the Multifile header.
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
bool scan_directory(vector_string &contents) const
Attempts to open the named filename as if it were a directory and looks for the non-hidden files with...
This is our own Panda specialization on the default STL set.
bool is_subfile_text(int index) const
Returns true if the indicated subfile represents text data, or false if it represents binary data.
std::string add_subfile(const std::string &subfile_name, const Filename &filename, int compression_level)
Adds a file on disk as a subfile to the Multifile.
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
bool exists() const
Returns true if the filename exists on the disk, false otherwise.
static Filename from_os_specific(const std::string &os_specific, Type type=T_general)
This named constructor returns a Panda-style filename (that is, using forward slashes,...