Tony Duckles [Sun, 23 Sep 2012 03:19:40 +0000 (22:19 -0500)]
Misc documentation tweaks
* svn2svn/run/svnreplay.py (main): Include mention of arguments
passed to the before-commit script.
* svn2svn/run/svnreplay.py (do_svn_add): Re-word comment for
clarity.
Tony Duckles [Tue, 21 Aug 2012 04:05:51 +0000 (23:05 -0500)]
Support before-commit hook option (--pre-commit)
Allow the caller to optionally run a shell script before each
replayed commit. This could be useful if you wanted to modify the
replayed file/directory contents, e.g. strip-out a certain file at a
certain source revision, etc.
* svn2svn/run/svnreplay.py (commit_from_svn_log_entry): Support
beforecommit hook.
* tests/t0103*: Add unit-tests for new --pre-commit option.
* svn2svn/run/svnreplay.py (main): Minor tweaks to command-line
argument help-text.
* svn2svn/shell.py (_run_raw_command): Fix typo
Tony Duckles [Tue, 14 Aug 2012 01:57:57 +0000 (20:57 -0500)]
Purge tests/_wc_* during "make clean"
As part of the "clean" Makefile target, purge any _wc_* directories
in the tests/ folder. Each of the individual unit-test scripts
*should* take care of purging these _wc_* folders, but include in
the Makefile just as a catch-all.
Tony Duckles [Fri, 10 Aug 2012 03:38:06 +0000 (22:38 -0500)]
Create _repo_ref if needed
Add Makefile targets and prereq's to call make-ref-repo.sh to build
_repo_ref if it doesn't already exist. This way, "make all" in a
freshly-cloned Git copy will automagically build the _repo_ref repo,
which is needed for the svnreplay-based tests.
Tony Duckles [Fri, 10 Aug 2012 03:32:56 +0000 (22:32 -0500)]
Move shared code to replay-lib.sh
* tests/replay-lib.sh (init_replay_repo): Move duplicated code from
t01??*.sh scripts to create an empty repo for svnreplay to this
shared function.
* tests/t010?*.sh: Use replay-lib.sh and init_replay_repo().
* tests/t010?*.sh: Respect $TEST_DIRECTORY shell variable rather
than hard-coding `pwd`.
Tony Duckles [Fri, 10 Aug 2012 03:26:07 +0000 (22:26 -0500)]
test-lib: test_expect_failure() expected failure is ok
The Git test-lib test_expect_failure() behavior is to call separate
test_known_ok/test_known_broken_failure helper tags which puts the
"known failure" results into separate buckets. This is the opposite
of what I want. I want test_expect_failure() to be the true-opposite
of test_expect_success(): if we expect a test to fail (i.e. exit 1),
then say the test passed if the command failed.
* tests/test-lib-functions.sh (test_expect_failure): Change
test_expect_failure() to be the true-opposite of
test_expect_success(): test succeeds if command exits with error
code.
Tony Duckles [Wed, 8 Aug 2012 03:00:26 +0000 (22:00 -0500)]
Introduce 'svnancest' utility
Introduce a helper utility for calling find_svn_ancestors(), to show
full ancestry ("svn copy" history) for a given starting URL
(optionally at a specific revision).
* svn2svn/run/svnacest.py: Adding
* svn2svn/run/common.py (in_svn, is_child_path, join_path,
find_svn_ancestors): Move common helper functions to new separate
library, for sharing with svn2svn/run/svnacest.py.
* svn2svn/svnclient.py (valid_svn_actions): Move this constant here
from svn2svn/run/svnreplay.py.
Tony Duckles [Sun, 5 Aug 2012 04:04:01 +0000 (23:04 -0500)]
Rename main utility from 'svn2svn.py' to 'svnreplay.py'
Rename the main utility executable for several reasons:
* Avoid overlapping with the 'svn2svn' module subdirectory.
* Make way for having other support utilities in the top-level folder.
Tony Duckles [Wed, 11 Apr 2012 13:21:31 +0000 (08:21 -0500)]
Proper quote-wrapping in test scripts
Wrap all path-variable references in double-quotes, so that if `pwd`
contains spaces bash will interpret things correctly. For URLs,
replace spaces with %20, though that's the extent of the
URL-encoding we try to do in these simple shell scripts.
Tony Duckles [Thu, 12 Apr 2012 01:46:34 +0000 (20:46 -0500)]
Better "svn list" handling
* svn2svn/svnclient.py (list, _parse_svn_list_xml): Add a tag for
proper "svn list --xml" handling.
* svn2svn/run/svn2svn.py: Remove get_svn_dirlist() in favor of using
svnclient.list(). Replace all usages of run_svn("svn list") with
svnclient.list(), which eliminates all hacky .split("\n") cases.
* svn2svn/run/svn2svn.py (verify_commit): Move unconditioned "OK"
ui.status line behind an else, so that "OK" message isn't
incorrectly logged when "FAIL" message is logged.
Tony Duckles [Thu, 12 Apr 2012 00:33:46 +0000 (19:33 -0500)]
Use "--ignore-externals" for "svn update" and "svn export"
* svn2svn/svnclient.py (update/remove/export): Introduce wrapper
functions for update/remove/export. For update and remove, pass
"--ignore-externals".
* svn2svn/svnclient.py: Rename private functions to have a leading
underscore.
* svn2svn/svnclient.py: Rename public functions to match svn
sub-command name: get_svn_info -> info, get_svn_status -> status,
get_prop_val -> propget, etc.
Tony Duckles [Mon, 9 Apr 2012 00:45:30 +0000 (19:45 -0500)]
Correctly handle source_url/target_url with chars needing URL-encoding
Use urllib.unquote() to URL-decode source_url/target_url values.
All URLs passed to run_svn() should go through svnclient.safe_path()
and we don't want to end-up *double* urllib.quote'ing if the user-
supplied source/target URL's are already URL-encoded or have chars
which need URL-encoding.
* svn2svn/run/svn2svn.py (real_main): Maintain source_url/target_url
as URL-decoded values.
* svn2svn/run/svn2svn.py (gen_tracking_revprops, build_rev_map):
Translate "internal" source_url back to an "external" (URL-encoded)
value.
* svn2svn/svnclient (parse_svn_info_xml): URL-decode 'url' and
'repos_url' values, since all downstream consumers should want these
to be in the "internal" (URL-decoded) format.
* tests/make-ref-repo.sh: Add test-cases to further stress-test
URL-encoding/decoding.
Tony Duckles [Sat, 7 Apr 2012 18:21:03 +0000 (13:21 -0500)]
Better path and URL encoding/escaping
* svn2svn/svnclient.py (safe_path): Creating to abstract building
paths which are safe to pass as svn command-line args. For URL's,
use urllib.quote() to URL-encode paths. For local paths, if file/dir
name includes "@", add a trailing "@" so svn doesn't get confused
about unintended text being parsed as a peg-revision.
* svn2svn/svnclient.py: Use safe_path() for all local and URL paths.
* svn2svn/run/svn2svn.py: Use svnclient.safe_path() for all local
and URL paths.
Tony Duckles [Sun, 25 Mar 2012 13:39:17 +0000 (08:39 -0500)]
Change verify-mode to report on all errors before raising exception
* svn2svn/run/svn2svn.py (verify_commit): Always finish a full pass,
writing out any error messages, and raise the VerificationError
exception at the end if needed.
* svn2svn/run/svn2svn.py (verify_commit): Make less verbose by
default. Show a progress message every 500 paths rather than logging
a message for every path.
Tony Duckles [Sun, 25 Mar 2012 04:49:10 +0000 (23:49 -0500)]
Fix edge-case problem with continue handling for --keep-revnum mode
* svn2svn/run/svn2svn.py (real_main): Check for --keep-revnum
problem condition inside main loop rather than ahead, so that we
compare the last target_url revision to the *next* source_url rev to
be replayed, not the *last* source_url rev that was replayed.
* svn2svn/run/svn2svn.py (keep_revnum): Use BreakHandler to ensure
that "Committed revision ..." message is always shown if we ran a
keep-revnum mode "svn commit".
* svn2svn/run/svn2svn.py (real_main): Fix "Finished at source
revision ..." message to be more accurate, to display the last
source_url rev replayed. Previously, in some cases (e.g. keep-revnum
mode) this could display the *next* source_url rev to be replayed.
Tony Duckles [Sun, 25 Mar 2012 04:26:02 +0000 (23:26 -0500)]
Don't allow starting a fresh (non-continue) replay into a non-empty target directory (by default)
* svn2svn/run/svn2svn.py (real_main): Prevent user from starting a
fresh replay into a non-empty target directory, since chances are
they wanted to do a continue instead. Introduce "--force" arg to
allow replaying into a non-empty target directory.
Tony Duckles [Sun, 25 Mar 2012 04:21:41 +0000 (23:21 -0500)]
Don't use parser.error() for error messages
* svn2svn/run/svn2svn.py (real_main): Don't use parser.error() for
error messages.
* svn2svn/run/svn2svn.py (commit_from_svn_log_entry): Include source
revision # in "Committed revision ..." message, so output is still
useful in non-verbose mode.
* svn2svn/run/svn2svn.py (real_main): Rename "_tmp_wc_target" to
* "_wc_target_tmp" (keep-revnum temp WC), for "_wc_*" matching.
Tony Duckles [Sat, 24 Mar 2012 20:15:19 +0000 (15:15 -0500)]
Be more quiet by default
* svn2svn/run/svn2svn.py (disp_svn_log_summary, real_main): Only
display source rev info for --verbose mode.
* svn2svn/run/svn2svn.py (commit_from_svn_log_entry): Move
"Committed revision %s." message down after the post-commit revprops
have been updated, since the commit+revprops are an atomic unit.
Tony Duckles [Sat, 24 Mar 2012 20:05:33 +0000 (15:05 -0500)]
For options.keep_author, update via post-commit rev-prop
Rather than using "svn commit --username %author%", mirror the
source author info by setting the "svn:author" revprop post-commit
the same as we do for "svn:date" (options.keep_date). This frees us
up to be able to use --username in the future for legitimate
source/target repo auth.
* svn2svn/run/svn2svn.py (commit_from_svn_log_entry): Update
"svn:author" via post-commit revprop rather than --username.
* README.mkd: Reflect new "svn:author" handling.
Tony Duckles [Sat, 24 Mar 2012 19:31:26 +0000 (14:31 -0500)]
Prevent KeyboardInterrupt's during SVN commit
* svn2svn/run/breakhandler.py: Adding
* svn2svn/run/svn2svn.py (commit_from_svn_log_entry): Use
BreakHandler to ensure that "svn commit" and post-commit rev-prop
updating happen as an atomic unit.
Tony Duckles [Sat, 24 Mar 2012 18:45:29 +0000 (13:45 -0500)]
Verify-mode
Add command-line args for verifying content and ancestry, i.e. for
self-testing the replay. Compare the source vs. target history to
make sure we ended-up with equivalent target revisions for each
applicable source revision.
* svn2svn/run/svn2svn.py (verify_commit): Adding.
* tests/make-ref-repo.sh: Add tests for source revisions which
we'll never have an equivalent target revision for, e.g.
svn:mergeinfo property changes.
Tony Duckles [Fri, 16 Mar 2012 03:23:08 +0000 (22:23 -0500)]
More fixes to iter_svn_log_entries ancestry-handling
* svn2svn/svnclient.py (iter_svn_log_entries): Correctly use and
respect ancestors() array: look for the next copyfrom_rev, and once
we crawl past the last copy-from then start at the final
path+revision.
* svn2svn/run/svn2svn.py (process_svn_log_entry): For action='R'
don't run "svn remove" command if path_offset="", i.e. don't try to
remove the root of the WC.
Change return format of find_svn_ancestors() to include
copyfrom_path+copyfrom_rev info. We need this in iter_svn_log_entries()
for correctly "svn log"'ing over only valid ancestry. Previously, we
were basically assuming that copyfrom_rev was always revision-1, and
trying to "svn log" on that revision-range can yield errors.
* svn2svn/run/svn2svn.py (find_svn_ancestors): Change return format
so that each entry in the array is a path+revision+copyfrom_path+copyfrom_rev
tuplet.
* svn2svn/run/svn2svn.py (do_svn_add): Minor changes to support new
find_svn_ancestors() return format.
* svn2svn/run/svn2svn.py (do_svn_add_dir): Fix local-not-remote
handling, to correctly handling directories.
Tony Duckles [Thu, 15 Mar 2012 01:13:32 +0000 (20:13 -0500)]
Correctly crawl source_url's ancestry back to origin, if any
* svn2svn/svnclient.py (iter_svn_log_entries): Support new optional
'ancestors' param. Use that to correctly grab revision-chunks,
following the ancestry history.
* svn2svn/run/svn2svn.py (join_path, in_ancestors): New functions.
* svn2svn/run/svn2svn.py (real_main): Use find_svn_ancestors() to get ancestry
of source_url and pass that to downstream functions.
Tony Duckles [Tue, 6 Mar 2012 04:01:28 +0000 (22:01 -0600)]
Refactor find_svn_ancestors() to make more generic
* svn2svn/run/svn2svn.py (find_svn_ancestors): Make 'stop_base_path'
optional so this function can be used generally to get the full
ancestry history.
* svn2svn/svnclient.py (get_first_svn_log_entry, get_last_svn_log_entry):
Add optional 'stop_on_copy' param and pass through to get_one_svn_log_entry().
Tony Duckles [Sat, 18 Feb 2012 22:35:04 +0000 (16:35 -0600)]
Fix process_svn_log_entry() to not include action="M" paths in skip_paths, so that do_svn_add() will try to check+create correct ancestry.
Update process_svn_log_entry() to calculate 'kind' if kind='none'.
Tony Duckles [Wed, 15 Feb 2012 20:48:14 +0000 (14:48 -0600)]
Fix a place in find_svn_ancestors() which should've been using is_child_path().
Display progress messages during build_rev_map(), for target repo's with lots of target_url commits to check.
Tony Duckles [Thu, 9 Feb 2012 03:25:24 +0000 (21:25 -0600)]
Update process_svn_log_entry() to calculate d['kind'] if missing.
Back-out changes from parse_svn_log_xml() to auto-calculate d['kind'] if not returned by "svn log --xml", since not all callers need d['kind']; just calculate on-demand where needed.
Tony Duckles [Thu, 9 Feb 2012 03:11:43 +0000 (21:11 -0600)]
Use is_child_path() for correct path-is-child-of-parent-path checking.
Add all changed_path's to commit_paths; don't use add_path() anymore.
Don't use add_path() for managing commit_paths
Tony Duckles [Sat, 4 Feb 2012 04:34:47 +0000 (22:34 -0600)]
Change iter_svn_log_entries() back to getting smaller chunks, for better performance on huge repositories.
Clarify safe iter_svn_log_entries() use-cases.
Use iter_svn_log_entries() where possible.
Tony Duckles [Tue, 24 Jan 2012 05:31:02 +0000 (23:31 -0600)]
* Use ui.status for all status messages (verbose and debug)
* Show "Committed revision" after commits to target repo
* Fixes to in_svn()
* Show "svn status" results before cleanup when we catch an external-command exception