[perl.git] branch nicholas/hv_h_split, created. v5.17.8-232-g0b340fe

[Nicholas Clark]

In perl.git, the branch nicholas/hv_h_split has been created

<http://perl5.git.perl.org/perl.git/commitdiff/0b340fe025421621146aac814778d80ad1df28c2?hp=0000000000000000000000000000000000000000>

at 0b340fe025421621146aac814778d80ad1df28c2 (commit)

- Log -----------------------------------------------------------------
commit 0b340fe025421621146aac814778d80ad1df28c2
Author: Yves Orton <***@gmail.com>
Date: Wed Feb 20 07:33:49 2013 +0100

Do not hard code the list of perl header files - discover them from disk instead

Includes a version bump to all the EU::MM files (to prevent any possible
$VERSION consistency checks from failing like they would on *nix)

This patch will be submitted to the EU::MM project in a modified form
and once merged there should replace these changes

M cpan/ExtUtils-MakeMaker/lib/ExtUtils/Liblist.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_AIX.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Any.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_BeOS.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Cygwin.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_DOS.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Darwin.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_MacOS.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_NW5.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_OS2.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_QNX.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_UWIN.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Unix.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_VMS.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_VOS.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Win32.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Win95.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MY.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/Mkbootstrap.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/Mksymlists.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/testlib.pm

commit a5ecd7a90df485373751ca4666f27f560f3c7305
Author: Yves Orton <***@gmail.com>
Date: Tue Feb 19 13:25:53 2013 +0100

semi-Randomize hash insertion order

M hv.c

commit edf686d97c62cd59540015e39472f7f15ec5b08d
Author: Yves Orton <***@gmail.com>
Date: Tue Feb 19 12:14:44 2013 +0100

Ensure that Module::Pluggable returns plugins in a predictable order

As of 5.18 hash traversal order will vary per-hash, and indeed more
frequently. The likelihood this results in one of the tests in M::P
failing becomes very high.

In general the test expectations were fragile, but probably almost
never failed. With hash traversal randomization they will fail
regularly.

Note the use a temporary array in part of this patch is because
sort in scalar context is undefined.

M cpan/Module-Pluggable/lib/Module/Pluggable.pm
M cpan/Module-Pluggable/lib/Module/Pluggable/Object.pm
M cpan/Module-Pluggable/t/19can_ok_clobber.t

commit 36db47b8c164fc825d2c640353ab59b1c08ad149
Author: Yves Orton <***@gmail.com>
Date: Tue Feb 19 12:12:34 2013 +0100

Fix code which modifies a hash while each()ing over it

It has never been correct to modify a hash during traversal by each().

In older perls this was unlikely to break the tests. In newer perls
it is relatively likely, and in 5.18 it will be almost certain.

This includes a version bump. Needs to be merged upstream.

M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Unix.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm
M cpan/ExtUtils-MakeMaker/t/basic.t

commit 423b89f0c0604b17b11eb5c641a91e236e6cc5ba
Author: Yves Orton <***@gmail.com>
Date: Mon Feb 18 16:13:08 2013 +0100

reset HvRAND() every hsplit, and properly rotate the bucket_rand. :-)

M hv.c

commit c50c6e235438112aa8486aa5b5ea20a5f92c1ed6
Author: Yves Orton <***@gmail.com>
Date: Mon Feb 18 10:18:48 2013 +0100

randomize bucket split insertion in hsplit()

M hv.c

commit 8a3c1392930e5f2cc6b9684424637600fa3d79d5
Author: Yves Orton <***@gmail.com>
Date: Mon Feb 18 09:36:35 2013 +0100

turn the ptr_hash inline code into a true sub

Borrowed from autobox, which is released under the same terms as Perl.

M embed.fnc
M embed.h
M hv.c
M proto.h

commit 690198e1ffd5c7c205ba38c7ee4f81253f02cb9f
Author: Yves Orton <***@gmail.com>
Date: Mon Feb 18 08:11:18 2013 +0100

Enable hash traversal order randomization

This patch causes Perl to use a random order per hash when traversing hashes via keys(), values() and each().

The random order is determined by add a new property xhv_rand to struct xpvhv. This property
is calculated on per traverse of the hash by using a pointer/integer hash function on the
address of the bucket array, and then XORing it with the bucket index counter during traverse.
This causes each hash to have a different order for traversing the buckets.

The purpose of this change is to make hash seed discovery attacks harder. An attacker loses
a lot of information when the keys are "randomized".

M hv.c
M hv.h
M t/op/smartkve.t

commit e872f0c498ac46f395ec57ec36cb2abc7d8d0519
Author: Yves Orton <***@gmail.com>
Date: Mon Feb 18 07:56:25 2013 +0100

Fix test fails due to hash traversal randomization

One cannot assume two hashes with the same keys will have the
the same key order. And after hash traversal randomization one
can assume they WONT.

M cpan/Pod-Simple/t/closeys.t

commit 727ab6a26803362d56ec6b978f755dabcd6e8876
Author: Yves Orton <***@gmail.com>
Date: Mon Feb 18 06:37:29 2013 +0100

Fix tests that fail due to hash traversal randomization

One cannot assume two hashes with the same keys will have the
the same key order. And after hash traversal randomization one
can assume they WONT.

M cpan/JSON-PP/t/019_incr.t

commit ff39733bc6682829331839e6b07f3809419500a6
Author: Yves Orton <***@gmail.com>
Date: Sun Feb 17 16:39:23 2013 +0100

Fix failing porting tests by bumping version number and tweaking pod

M ext/Hash-Util/lib/Hash/Util.pm

commit ba711ae5b4185dada66f9d47235206791dd07c9a
Author: Yves Orton <***@gmail.com>
Date: Tue Feb 12 05:06:48 2013 +0100

default to PERL_FUNC_ONE_AT_A_TIME_HARD for all builds

For testing, but maybe for ever

M hv_func.h

commit 54a42a0b98387647e09151061580b4de8c13fd34
Author: Yves Orton <***@gmail.com>
Date: Mon Feb 11 19:33:00 2013 +0100

Remove XXHash (for now)

For some reason I could never get it working anyway, due to strange
segfaults that I did not have the time to track down.

M hv_func.h

commit 1dc3e571e3c4ecf6e679588cd47ba7baaa70a713
Author: Yves Orton <***@gmail.com>
Date: Mon Jan 21 17:11:31 2013 +0100

random tweaks

M hv_func.h

commit dd07797f306e12c9fd1c21050a3dae731752b6e2
Author: Yves Orton <***@gmail.com>
Date: Wed Dec 12 00:13:53 2012 +0100

fixup wording

M ext/Hash-Util/lib/Hash/Util.pm

commit 68e363f68bc2ec927f4d9fc18324d032ed64971a
Author: Yves Orton <***@gmail.com>
Date: Tue Dec 11 23:46:37 2012 +0100

add a "hash quality score" to Hash::Util::bucket_stats()

M ext/Hash-Util/lib/Hash/Util.pm

commit 8ac94e1c1b9741d541836ad3992be2b267a5766d
Author: Yves Orton <***@gmail.com>
Date: Tue Dec 11 08:50:58 2012 +0100

Attempt to eliminate a signed mistmatch in comparison warning

as far as I can tell 'i' can only be positive here.

M hv_func.h

commit 7382a7f28c0ff9c6ff9a015fe03601ac7a0017be
Author: Yves Orton <***@gmail.com>
Date: Mon Dec 10 09:43:59 2012 +0100

include missing headers so XS modules rebuild when they change

M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Unix.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm

commit dcd75883353e0a188bfba5f1fd8da5df4cd6f45c
Author: Yves Orton <***@gmail.com>
Date: Mon Dec 10 08:36:43 2012 +0100

add a hardened one-at-a-time hash variant

Mix in additional randomness into the final value.

M hv_func.h

commit d64439dd8dfb12593fb0fc1445a12ed5b8b40d85
Author: Yves Orton <***@gmail.com>
Date: Mon Dec 10 00:40:43 2012 +0100

add xxhash

M hv_func.h

commit b63ba01573a6373e10db0cc3c17db59d92da6aa7
Author: Yves Orton <***@gmail.com>
Date: Sun Dec 9 17:01:46 2012 +0100

Split out hash functions into new file and turn into inline static functions

This includes various tweaks related to building SipHash.

M Cross/Makefile-cross-SH
M MANIFEST
M Makefile.SH
M Makefile.micro
M NetWare/Makefile
M configpm
M hv.h
A hv_func.h
M win32/Makefile
M win32/Makefile.ce

commit ea080faae2b2da4081a778d7d44c8b291457aa8c
Author: Yves Orton <***@gmail.com>
Date: Sat Dec 8 16:26:01 2012 +0100

remove toy code

M hv.h

commit 69bffcbb063e0be1b744c6a225d8e7e603e360a9
Author: Yves Orton <***@gmail.com>
Date: Sat Dec 8 16:24:06 2012 +0100

remove unused define

M hv.h
-----------------------------------------------------------------------

--
Perl5 Master Repository

[demerphq]

Note Thatcher i plan to rebelse/squash a lot of THE crap out of thuis
commit sequence before i merge.

Yves on an ipad

Post by Nicholas Clark
In perl.git, the branch nicholas/hv_h_split has been created
<
http://perl5.git.perl.org/perl.git/commitdiff/0b340fe025421621146aac814778d80ad1df28c2?hp=0000000000000000000000000000000000000000
at 0b340fe025421621146aac814778d80ad1df28c2 (commit)
- Log -----------------------------------------------------------------
commit 0b340fe025421621146aac814778d80ad1df28c2
Date: Wed Feb 20 07:33:49 2013 +0100
Do not hard code the list of perl header files - discover them from disk instead
Includes a version bump to all the EU::MM files (to prevent any possible
$VERSION consistency checks from failing like they would on *nix)
This patch will be submitted to the EU::MM project in a modified form
and once merged there should replace these changes
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/Liblist.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_AIX.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Any.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_BeOS.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Cygwin.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_DOS.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Darwin.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_MacOS.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_NW5.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_OS2.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_QNX.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_UWIN.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Unix.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_VMS.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_VOS.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Win32.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Win95.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MY.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/Mkbootstrap.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/Mksymlists.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/testlib.pm
commit a5ecd7a90df485373751ca4666f27f560f3c7305
Date: Tue Feb 19 13:25:53 2013 +0100
semi-Randomize hash insertion order
M hv.c
commit edf686d97c62cd59540015e39472f7f15ec5b08d
Date: Tue Feb 19 12:14:44 2013 +0100
Ensure that Module::Pluggable returns plugins in a predictable order
As of 5.18 hash traversal order will vary per-hash, and indeed more
frequently. The likelihood this results in one of the tests in M::P
failing becomes very high.
In general the test expectations were fragile, but probably almost
never failed. With hash traversal randomization they will fail
regularly.
Note the use a temporary array in part of this patch is because
sort in scalar context is undefined.
M cpan/Module-Pluggable/lib/Module/Pluggable.pm
M cpan/Module-Pluggable/lib/Module/Pluggable/Object.pm
M cpan/Module-Pluggable/t/19can_ok_clobber.t
commit 36db47b8c164fc825d2c640353ab59b1c08ad149
Date: Tue Feb 19 12:12:34 2013 +0100
Fix code which modifies a hash while each()ing over it
It has never been correct to modify a hash during traversal by each().
In older perls this was unlikely to break the tests. In newer perls
it is relatively likely, and in 5.18 it will be almost certain.
This includes a version bump. Needs to be merged upstream.
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Unix.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm
M cpan/ExtUtils-MakeMaker/t/basic.t
commit 423b89f0c0604b17b11eb5c641a91e236e6cc5ba
Date: Mon Feb 18 16:13:08 2013 +0100
reset HvRAND() every hsplit, and properly rotate the bucket_rand. :-)
M hv.c
commit c50c6e235438112aa8486aa5b5ea20a5f92c1ed6
Date: Mon Feb 18 10:18:48 2013 +0100
randomize bucket split insertion in hsplit()
M hv.c
commit 8a3c1392930e5f2cc6b9684424637600fa3d79d5
Date: Mon Feb 18 09:36:35 2013 +0100
turn the ptr_hash inline code into a true sub
Borrowed from autobox, which is released under the same terms as Perl.
M embed.fnc
M embed.h
M hv.c
M proto.h
commit 690198e1ffd5c7c205ba38c7ee4f81253f02cb9f
Date: Mon Feb 18 08:11:18 2013 +0100
Enable hash traversal order randomization
This patch causes Perl to use a random order per hash when traversing
hashes via keys(), values() and each().
The random order is determined by add a new property xhv_rand to
struct xpvhv. This property
is calculated on per traverse of the hash by using a pointer/integer
hash function on the
address of the bucket array, and then XORing it with the bucket index
counter during traverse.
This causes each hash to have a different order for traversing the buckets.
The purpose of this change is to make hash seed discovery attacks
harder. An attacker loses
a lot of information when the keys are "randomized".
M hv.c
M hv.h
M t/op/smartkve.t
commit e872f0c498ac46f395ec57ec36cb2abc7d8d0519
Date: Mon Feb 18 07:56:25 2013 +0100
Fix test fails due to hash traversal randomization
One cannot assume two hashes with the same keys will have the
the same key order. And after hash traversal randomization one
can assume they WONT.
M cpan/Pod-Simple/t/closeys.t
commit 727ab6a26803362d56ec6b978f755dabcd6e8876
Date: Mon Feb 18 06:37:29 2013 +0100
Fix tests that fail due to hash traversal randomization
One cannot assume two hashes with the same keys will have the
the same key order. And after hash traversal randomization one
can assume they WONT.
M cpan/JSON-PP/t/019_incr.t
commit ff39733bc6682829331839e6b07f3809419500a6
Date: Sun Feb 17 16:39:23 2013 +0100
Fix failing porting tests by bumping version number and tweaking pod
M ext/Hash-Util/lib/Hash/Util.pm
commit ba711ae5b4185dada66f9d47235206791dd07c9a
Date: Tue Feb 12 05:06:48 2013 +0100
default to PERL_FUNC_ONE_AT_A_TIME_HARD for all builds
For testing, but maybe for ever
M hv_func.h
commit 54a42a0b98387647e09151061580b4de8c13fd34
Date: Mon Feb 11 19:33:00 2013 +0100
Remove XXHash (for now)
For some reason I could never get it working anyway, due to strange
segfaults that I did not have the time to track down.
M hv_func.h
commit 1dc3e571e3c4ecf6e679588cd47ba7baaa70a713
Date: Mon Jan 21 17:11:31 2013 +0100
random tweaks
M hv_func.h
commit dd07797f306e12c9fd1c21050a3dae731752b6e2
Date: Wed Dec 12 00:13:53 2012 +0100
fixup wording
M ext/Hash-Util/lib/Hash/Util.pm
commit 68e363f68bc2ec927f4d9fc18324d032ed64971a
Date: Tue Dec 11 23:46:37 2012 +0100
add a "hash quality score" to Hash::Util::bucket_stats()
M ext/Hash-Util/lib/Hash/Util.pm
commit 8ac94e1c1b9741d541836ad3992be2b267a5766d
Date: Tue Dec 11 08:50:58 2012 +0100
Attempt to eliminate a signed mistmatch in comparison warning
as far as I can tell 'i' can only be positive here.
M hv_func.h
commit 7382a7f28c0ff9c6ff9a015fe03601ac7a0017be
Date: Mon Dec 10 09:43:59 2012 +0100
include missing headers so XS modules rebuild when they change
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Unix.pm
M cpan/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm
commit dcd75883353e0a188bfba5f1fd8da5df4cd6f45c
Date: Mon Dec 10 08:36:43 2012 +0100
add a hardened one-at-a-time hash variant
Mix in additional randomness into the final value.
M hv_func.h
commit d64439dd8dfb12593fb0fc1445a12ed5b8b40d85
Date: Mon Dec 10 00:40:43 2012 +0100
add xxhash
M hv_func.h
commit b63ba01573a6373e10db0cc3c17db59d92da6aa7
Date: Sun Dec 9 17:01:46 2012 +0100
Split out hash functions into new file and turn into inline static functions
This includes various tweaks related to building SipHash.
M Cross/Makefile-cross-SH
M MANIFEST
M Makefile.SH
M Makefile.micro
M NetWare/Makefile
M configpm
M hv.h
A hv_func.h
M win32/Makefile
M win32/Makefile.ce
commit ea080faae2b2da4081a778d7d44c8b291457aa8c
Date: Sat Dec 8 16:26:01 2012 +0100
remove toy code
M hv.h
commit 69bffcbb063e0be1b744c6a225d8e7e603e360a9
Date: Sat Dec 8 16:24:06 2012 +0100
remove unused define
M hv.h
-----------------------------------------------------------------------
--
Perl5 Master Repository