An obvious way to get parallel streams of PRNs is to partition a
sequence of a given pseudorandom number generator into suitable subsequences.
This can be achieved in different ways.
In Subsection , we study the well known ``leapfrog'' technique where the original sequence of a generator is split into lagged subsequences with a given lag . Whereas a single stream of a LCG performs well in practice, due to good spectral test results, the quality of lagged subsequences of such a stream may be worsen considerably. Hence whenever lagged subsequences occur in a particular simulation problem these sequences have to be tested as well.
In [65] we showed how to analyze correlations within and between lagged subsequences with arbitrary step sizes generated from LCGs with poweroftwo and prime moduli. In Subsection we recall basic concepts from [65] and apply them to several examples of LCGs.
By varying the initial seed one easily may partition a sequence of a LCG into consecutive blocks of a given length. An extensive discussion of this method for LCGs with respect to parallelization was given by DeMatteis and Pagnutti [39] (see also [38,40,41,42,43,44,53]). It turns out that only reduced fractions of sequences produced by LCGs can safely be used because of the well known longrange correlations which may cause high correlations between parallel streams. Critical lengths of consecutive blocks for several LCGs are given e.g. for RANF [38,40,42], L'Ecuyer LCG 3 [38], MINSTD [38], SuperDuper and RANDU [42]. Using the spectral test it is possible to exhibit these longrange correlations in a stronger way. Whereas the latter authors studied correlations between pairs of processors only, the spectral test offers an easy tool to analyze correlations between an arbitrary number of parallel streams, see Subsection .
Sometimes it is recommended, to use separate generators to produce parallel streams of random numbers e.g. see [8]. In Subsection we give a simple example of three LCGs with perfect spectral test results but strong correlations within the parallel streams obtained from these generators.
Our subsequence analyses of LCGs use classical notions from linear random number generation. A more general analysis of lagged subsequences with arbitrary lacunary indices from multiple recursive generators has been discussed and performed in [136,148].
With the ``leapfrog'' technique [16,27,54,132,134]
the output
of a LCG is partitioned into lagged subsequences of the form
Consider a mixed linear congruential generator with
maximal period . For this
generator, the subsequence () is also produced by (see also
[134,203])
Even if one chooses subsequences with maximal period , these sequences can be of inferior quality. This can be easily shown by the spectral test. The table below gives some examples. G_k denotes the LCG which generates a fullperiod subsequence of generator G. Further results can be found in [62,63].
Note that FISH15_23, FISH7_105 exhibit normalized spectral test results worse than RANDU. The zoom into the 3dimensional unit cube above stems from FISH15_23. The second graphics shows the 15 hyperplanes that cover all threedimensional overlapping tuples generated by FISH7_105. A parallel application using LCG.1 is given in [159].
If a subsequenceLCG does not have full period, the underlying lattice structure obviously can be of reduced quality as well. In the graphics below we demonstrate this behavior with some zooms into the unit square which contain all overlapping vectors generated by the by the LCGs: MINSTD_77, SIMSCRIPT_78 and RANF_36 . Compare the zooms in the Sections , and .
MINSTD_77  SIMSCRIPT_78  RANF_36 
But how to assess nonfull period subsequences with the spectral test?
As an example, we consider
``the reminder of the good old days, the early 1950s'',
the LCG with multiplier
(BCSLIB, URN12, CDC Computers, SIMULA)^{7}, which already was analyzed by
Coveyou and MacPherson [36] with the spectral test.
In particular we analyze certain nonfull period subsequences of
which, especially for , is
studied in different parallel settings (including the leapfrog method)
in [16].
Quote[16, p. 451]: This 'leapfrog' algorithm was subjected to an analysis of correlation, exactly as was done for the Lehmer tree algorithm. The choices , , and were again made and 50000 numbers were generated in each parallel sequence for analysis by linear regression. From 8 to 512 parallel processes were examined. No significant correlations were found between pairs of processes.
Let's make a more detailed study of these ``8 to 512 parallel processes''. In the subsections below we first recall some basic lattice theory of LCGs and then study the quality reduction of the lattice structure generated by lagged subsequences with step sizes , of . Note that this analysis applies to arbitrary linear congruential pseudorandom number generator with poweroftwo moduli as well.
We now describe the lattice structure obtained by subsequences ()
with step sizes , . Recall the basic recursion of :
, , and w.l.o.g. .
From [201, Prop. 1] (see also [73, p. 599]) we get
Therefore, the spectral test of the subsequences has to be calculated using the latter vector (with constant !).
Note that if one uses constant for the calculation, this would give the spectral test for the union of all subsequencegrids. This is not the appropriate way to asses the subsequence structure. For example, the GENsubsequence with step size yields the weak spectral test whereas for the union of the subsequencegrids .
We applied the normalized spectral test , , to subsequences with step sizes , . As can be seen from the tables below, with increasing , the quality of the subsequencelattices strongly decreases. The tables report the results for with moduli , , 35, 48. Perhaps there are no ``correlations'' between pairs of parallel streams, as stated above, but the quality of each stream is very poor for .
1  2  3  4  5  6  7  8  9  
2  0.8215  0.6721  0.8523  0.3516  0.8074  0.3232  0.6026  0.8581  0.6962  0.9306 
3  0.6130  0.3172  0.6175  0.8089  0.6096  0.8078  0.5735  0.3437  0.0541  0.0170 
4  0.4117  0.6233  0.2177  0.5172  0.5141  0.6538  0.0588  0.0699  0.0743  0.0442 
5  0.3466  0.7097  0.2594  0.6708  0.7395  0.2124  0.0922  0.1059  0.0942  0.0884 
6  0.6277  0.6898  0.4727  0.7297  0.4068  0.2283  0.1531  0.1719  0.1220  0.1370 
7  0.6686  0.6095  0.5495  0.7214  0.2995  0.3307  0.1690  0.1866  0.1682  0.1857 
8  0.4003  0.6781  0.6060  0.4915  0.3933  0.2808  0.2165  0.2361  0.2102  0.2292 
1  2  3  4  5  6  7  8  9  
2  0.5809  0.4897  0.6027  0.8904  0.7587  0.9141  0.8522  0.3773  0.8814  0.6853 
3  0.4145  0.8614  0.7296  0.8089  0.6924  0.6897  0.8175  0.6827  0.21651  0.0341 
4  0.8004  0.7402  0.6691  0.7393  0.3793  0.6151  0.2795  0.0415  0.0494  0.0526 
5  0.6401  0.6601  0.4333  0.8432  0.6032  0.1401  0.1609  0.0699  0.0803  0.0714 
6  0.6951  0.6746  0.5526  0.5160  0.4315  0.1614  0.1812  0.1216  0.1364  0.0969 
7  0.6379  0.6570  0.5627  0.6143  0.3855  0.2457  0.2293  0.1387  0.15309  0.1380 
8  0.7473  0.5229  0.7831  0.5852  0.4863  0.2165  0.1928  0.1821  0.1985  0.1768 
1  2  3  4  5  6  7  8  9  
2  0.9599  0.2776  0.5677  0.2635  0.6914  0.8752  0.9076  0.4763  0.6009  0.8353 
3  0.5761  0.7166  0.7645  0.6931  0.2755  0.6031  0.7606  0.2767  0.7722  0.6447 
4  0.5178  0.7922  0.5831  0.8606  0.6442  0.5691  0.4973  0.7558  0.6598  0.0988 
5  0.4799  0.6529  0.6434  0.3833  0.3943  0.5396  0.2998  0.2439  0.0350  0.0402 
6  0.6001  0.4908  0.5149  0.6316  0.3332  0.6300  0.1211  0.1359  0.0508  0.0571 
7  0.6956  0.7539  0.6962  0.5439  0.6390  0.3896  0.1297  0.1432  0.0913  0.1008 
8  0.6715  0.6452  0.6179  0.5873  0.6679  0.3292  0.1875  0.1524  0.0910  0.0993 
The same analysis as for was made for RANF (Sect. ) as well [64]. In the case of RANF, step sizes , are of special interest, since such step sizes are implemented on CRAY systems (e.g. see the Cray Math Library Reference Manual SR2080). Similar as for the lagged subsequences with step sizes 64, 128, 256, 512 show poor spectral test results.
For large poweroftwo step sizes we can
determine the spectral test for arbitrary multiplicative LCGs with
poweroftwo moduli . Let ,
.
Therefore the order of
in
the multiplicative group equals
.
From () and () we obtain
Now let . Equation () yields , and therefore . Further we observe that for and thus for the latter powers . Similarly, for the same range of , we have and therefore for .
Again consider . The table below verifies the calculations above. It exhibits the values and the normalized spectral tests for and step sizes , . Note that for we obtain . The spectral tests for and can also be verified theoretically in the same way as above (in this case we observe that ).


In Entacher [65] we showed how to analyze correlations within and between lagged subsequences with arbitrary step sizes generated from LCGs with poweroftwo and prime moduli. The main results of the latter paper are the following:
Consider the full period standard cases of LCGs: (i) is prime, is a primitive root modulo , , and , (ii) is a power of 2, , and c is odd, and finally (iii) and as in (ii) but and is odd [8,73,130,184,201].
In sharp contrast to LCGs with poweroftwo moduli, lagged subsequences from prime LCGs (i) exhibit no pure lattice structure in general. As examples, consider ,78,0,1) and ,114,0,1). The graphics on the left side in the figures below show the lattice structure of all overlapping vectors in dimension generated from (upper graphics) and , whereas the remaining plots show the structures obtained by the subsequences () with step sizes and . The spectral tests (as described below) are given in the figures as well.
,  ,  , 
,  ,  , 
Using the spectral test for the assessment of correlations within lagged subsequences () with step size generated from prime LCGs, one has to apply the lattice in the calculation of the test. But in the nonfullperiod case (), this strategy yields the assessment of the union of all overlapping vectors generated by each subsequence () with . As you may imagine from the figures above, additional correlation analysis should be performed for each single subsequence. Nevertheless, the latter lattice can be applied for finding (bad) subsequences and for a correlation analysis of consecutive blocks (), see Section below.
In order to analyze correlations between lagged subsequences of prime LCGs one has to assess the same set of vectors , as in Remark . Similar as for overlapping vectors of lagged subsequences from prime LCGs, these vectors exhibit no pure lattice structure in general. Therefore the spectral test is not the obvious measure to assess such correlations, exept of an analysis of fullperiod subsequences.
(9) 
(10) 
As an example we use GEN with modulus , and consider consecutive blocks of length , . The graphics below shows vectors , , , for block lengths , , respectively. Below the plots, the spectral tests and the estimates for the number of hyperplanes are given as well. Whereas the correlations between pairs of consecutive streams with decreasing block length rapidly become better, the spectral tests in dimension remain stable.
:
:
:
These graphics exhibit strong ``longrange'' correlations between consecutive streams with large block size. Such longrange correlations between pairs of consecutive blocks have already been studied by DeMatteis and Pagnutti et al. [39,41,38,42,43,44,59]. Using the spectral test to assess such correlations was already suggested by Durst [53] and related concepts can be found in [40,195] ^{8}. Similar as DeMatteis et. al. the latter authors applied their correlation analysis only in dimension two for computational and mathematical reasons.
In this section we will update the concepts mentioned above, i.e. we use the spectral test to extend the analysis of correlations between consecutive blocks of random numbers to higher dimensions . The following table shows a normalized spectral test analysis of consecutive blocks obtained from GEN. The table contains results of for block lengths , and dimensions . From the spectral test results we observe that, no practically relevant consecutive blocks (with poweroftwo block lengths) from GEN and related generators should be used in parallel.
Note that in contrast to the spectral tests in Subsection , the tables below exhibit spectral tests of the union of all grids produced from overlapping vectors of subsequences with step size .
12  13  14  15  16  17  18  19  
2  0.5189  0.7827  0.4101  0.4001  0.7036  0.7984  0.6388  0.5922  0.7505 
3  0.3252  0.6316  0.9157  0.8660  0.2165  0.0541  0.0135  0.0034  0.0008 
4  0.1662  0.0208  0.0026  0.0013  0.0013  0.0013  0.0013  0.0013  0.0013 
5  0.0116  0.0116  0.0044  0.0044  0.0044  0.0044  0.0044  0.0044  0.0044 
6  0.0202  0.0202  0.0121  0.0121  0.0121  0.0121  0.0121  0.0121  0.0121 
7  0.0413  0.0413  0.0191  0.0191  0.0191  0.0191  0.0191  0.0191  0.0191 
8  0.0455  0.0455  0.0322  0.0322  0.0322  0.0322  0.0322  0.0322  0.0322 
Using only odd block lengths (hence the block length does not divide the period of the generator which also implies that , has full period) provides a simple way against correlations between consecutive blocks obtained from poweroftwo LCGs. But previously testing is recommended in this case as well. The table below shows normalized spectral tests , , using for block lengths , . Similarly, as for the results below there are no conspicuous correlations for .
8  9  10  11  12  13  14  15  
2  0.5568  0.7586  0.3225  0.5233  0.9388  0.3400  0.4041  0.7146  0.7608 
3  0.7885  0.0649  0.7828  0.5840  0.7322  0.5855  0.3608  0.6067  0.8315 
4  0.5524  0.6749  0.7581  0.5583  0.5056  0.6959  0.1823  0.5925  0.4150 
5  0.6250  0.6372  0.5340  0.5056  0.7240  0.3753  0.1847  0.7383  0.5295 
6  0.7074  0.5841  0.7409  0.5772  0.5386  0.5517  0.1676  0.5208  0.6381 
7  0.6703  0.5663  0.6403  0.5384  0.6852  0.6355  0.3434  0.6801  0.6012 
8  0.7022  0.6511  0.4436  0.6024  0.5562  0.5624  0.4172  0.5401  0.4828 
For prime LCGs there are longrange correlations as well. As an example we consider MINSTD. Typical behavior of prime LCGs in longrange correlation analyses with the spectral test is shown in the table below^{9}. We derived the normalized spectral test , using MINSTD and lattice , , with block lengths , for . Note that equals the period of MINSTD. The table reports those results where at least one value , , turns out to be lower than . Column gives the prime factors of since from the results below one may assume that there is a strong impact between the prime factors of the period and those of critical distances (for more details see the theoretical verification below).
As a first consequence these results require to perform any longrange correlation analysis or calculations of critical distances [38,42,43,59] also in higher dimensions . Consider for example the case , which means that we partition the full period of into three consecutive blocks of length , each of which is used as a single stream in parallel. The spectraltest shows that there are strong correlations within all three streams, whereas pairs of these streams are not correlated due to the large value of . Further illustration is provided by the measure which in case equals and . Hence, overlapping three dimensional vectors lie in at most two hyperplanes whereas two dimensional vectors lie within a fine lattice structure, due to the large number of hyperplanes estimated in dimension two.
3  4  5  6  7  8  
2  0.00003  
3  0.8849  0.00120  
5  0.00014  0.00488  0.02762  0.07813  
6  0.8849  0.00120  0.00552  0.01562  0.03051  
7  0.6813  0.5699  0.4995  0.6901  0.7930  0.09129  
9  0.5974  0.6704  0.4079  0.5936  0.7703  0.05976  0.08347  
10  0.00689  0.2368  0.6535  0.3313  0.4088  0.26948  0.3764  
14  0.6813  0.5699  0.4995  0.6901  0.7930  0.09129  0.06816  
18  0.8906  0.3122  0.644  0.5656  0.8061  0.05976  0.08347  
30  0.9047  0.03419  0.19339  0.5470  0.4277  0.3684  0.5146  
31  0.3290  0.00557  0.03149  0.08908  0.1739  0.2782  0.3886  
62  0.00257  0.08839  0.5000  0.08908  0.1739  0.2782  0.3886  
80  0.05772  0.7298  0.8117  0.6029  0.3883  0.6211  0.7482  
83  0.06190  0.7229  0.8283  0.5640  0.6208  0.6901  0.7741  
93  0.5721  0.4139  0.5675  0.05063  0.09886  0.1581  0.2208  
124  0.8144  0.08451  0.3894  0.8419  0.4607  0.6409  0.7001  
155  0.01028  0.3536  0.5955  0.5829  0.4934  0.7303  0.5600  
251  0.07531  0.6519  0.5343  0.8393  0.5119  0.5488  0.7099  
297  0.09096  0.8742  0.7529  0.6269  0.3823  0.6114  0.5367  
358  0.07184  0.6342  0.6430  0.6453  0.6316  0.5574  0.5763  
374  0.09356  0.4917  0.5778  0.5288  0.6522  0.7707  0.3886 
For block lengths with small divisors of the period , we easily can verify the behavior of certain spectral test results in the table above. In this case the order of in the multiplicative group equals . Hence for prime numbers we get , and therefore the weak spectral test . Now consider nonprime divisors . For example let . Thus we get . Let . From we obtain . For dimension we use and , which yield . Similar calculations yield the entries of the following table which are valid for arbitrary primitive roots modulo , and hence for all full period multiplicative LCGs with prime modulus . The table below exhibits the values for .
3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  
2  2  
3  3  
6  3  2  2  2  
7  7  
9  3  3  3  
11  11  
14  7  2  2  2  2  2  2  2  
18  3  3  2  2  2  2  2  2  2  2  2  2 
For the situation slightly changes. In this case, the (bad) spectral test values depend on the explicit multiplier . For this multiplier we get and therefore , which yields the bad spectral tests in dimensions . Note that for an holds for all primitive roots . Hence, if equals a multiple of 31 such as we also obtain bad results.
From arbitrary separate generators, parallel streams of random numbers can be generated as well. This strategy is sometimes recommended in the literature e.g. see [8], or supported by parallel random number libraries [171]. In this section we want to demonstrate the danger of this strategy. As a simple example consider the following three LCGs: Let , , and , and denote the multiplicative LCG with modulus , seed and multiplier by . These multipliers are proposed in [171, Table 7] for use in parallel environments. was also proposed by Fishman, see Section . The normalized spectral tests of all three LCGs in the latter dimensions are given in the table below.


There are no conspicuous correlations between parallel streams generated from these LCGs as we may assume from the scatter plots of vectors and given in the following graphics.
But if we change multiplier and generate the same plots as before, we easily can exhibit strong correlations between parallel streams from these LCGs, see the graphics below. Note that the new multiplier yields perfect spectral test results and as well.