(前向)管道运算符可以/是否可以阻止尾部调用优化?

首要因素

对于工作中的参数优化问题,我写了一种遗传算法来找到一些好的设置,因为蛮力解决方案不可行。不幸的是,当我早上回来时,大多数时候都会看到StackOverflowException

我已经使用F#已有一段时间了,因此我知道TCO和具有累加器参数的函数的需求,并且通常使用该形式。

经过大量搜索之后,我认为我能够确定触发异常的代码:

breedPopulation alive |> simulate (generation + 1) lastTime ewma

breedPopulation从当前的个体中产生出新一代alive然后,通过调用来开始下一轮/一代simulate当我查看反汇编(总noob)时,我发现了一些pop和a ret,因此它看起来不像是常规(非尾部)的呼叫。

mov         rcx,qword ptr [rbp+10h]  
mov         rcx,qword ptr [rcx+8]  
mov         rdx,qword ptr [rbp-40h]  
cmp         dword ptr [rcx],ecx  
call        00007FFA3E4905C0  
mov         qword ptr [rbp-0F0h],rax  
mov         r8,qword ptr [rbp-0F0h]  
mov         qword ptr [rbp-80h],r8  
mov         r8,qword ptr [rbp-78h]  
mov         qword ptr [rsp+20h],r8  
mov         r8d,dword ptr [rbp+18h]  
inc         r8d  
mov         rdx,qword ptr [rbp+10h]  
mov         r9,qword ptr [rbp-20h]  
mov         rcx,7FFA3E525960h  
call        00007FFA3E4A5040  
mov         qword ptr [rbp-0F8h],rax  
mov         rcx,qword ptr [rbp-0F8h]  
mov         rdx,qword ptr [rbp-80h]  
mov         rax,qword ptr [rbp-0F8h]  
mov         rax,qword ptr [rax]  
mov         rax,qword ptr [rax+40h]  
call        qword ptr [rax+20h]  
mov         qword ptr [rbp-100h],rax  
mov         rax,qword ptr [rbp-100h]  
lea         rsp,[rbp-10h]  
pop         rsi  
pop         rdi  
pop         rbp  
ret

扔掉管道操作员并将饲养置于正常参数位置后,拆卸过程有所不同。

//    simulate (generation + 1) lastTime ewma (breedPopulation alive)
mov         ecx,dword ptr [rbp+18h]  
inc         ecx  
mov         dword ptr [rbp-30h],ecx  
mov         rcx,qword ptr [rbp-20h]  
mov         qword ptr [rbp-38h],rcx  
mov         rcx,qword ptr [rbp-80h]  
mov         qword ptr [rbp-0F0h],rcx  
mov         rcx,qword ptr [rbp+10h]  
mov         rcx,qword ptr [rcx+8]  
mov         rdx,qword ptr [rbp-48h]  
cmp         dword ptr [rcx],ecx  
call        00007FFA3E4605C0  
mov         qword ptr [rbp-0F8h],rax  
mov         rax,qword ptr [rbp-0F8h]  
mov         qword ptr [rbp+30h],rax  
mov         rax,qword ptr [rbp-0F0h]  
mov         qword ptr [rbp+28h],rax  
mov         rax,qword ptr [rbp-38h]  
mov         qword ptr [rbp+20h],rax  
mov         eax,dword ptr [rbp-30h]  
mov         dword ptr [rbp+18h],eax  
nop  
jmp         00007FFA3E47585B

这肯定会更短,最后的结果jmp甚至比尾声更好。

因此,我想了解似乎是问题的原因以及为什么|>会导致问题的发生;毕竟,这是多年以来第一次咬我。在什么情况下会发生这种情况,我们需要注意什么?


更新:盖伊指出我的清单不是IL而是汇编语言之后,我首先改写了这个问题。这是我在ILSpy中发现的

使用|>运算符

看着反编译的C#,代码似乎在两者之间来回跳转

internal static FSharpFunc<Types.Genome[], System.Tuple<System.Tuple<float, float>, LbpArea[]>[]> simulate@265-1(Universe x, System.Threading.ManualResetEvent pleaseStop, int generation, System.DateTime lastTime, FSharpOption<double> ewma)
{
    return new $Universe.simulate@267-2(x, pleaseStop, generation, lastTime, ewma);
}

// internal class simulate@267-2
public override System.Tuple<System.Tuple<float, float>, LbpArea[]>[] Invoke(Types.Genome[] population)
{
    LbpArea[][] array = ArrayModule.Parallel.Map<Types.Genome, LbpArea[]>(this.x.genomeToArray, population);
    FSharpFunc<System.Tuple<System.Tuple<float, float>, LbpArea[]>, float> accessFitness = this.x.accessFitness;
    System.Tuple<System.Tuple<float, float>, LbpArea[]>[] array2 = ArrayModule.Filter<System.Tuple<System.Tuple<float, float>, LbpArea[]>>(new $Universe.alive@274(accessFitness), ArrayModule.Parallel.Map<LbpArea[], System.Tuple<System.Tuple<float, float>, LbpArea[]>>(new $Universe.alive@273-1(this.x), array));
    if (array2 == null)
    {
        throw new System.ArgumentNullException("array");
    }
    System.Tuple<System.Tuple<float, float>, LbpArea[]>[] array3 = ArrayModule.SortWith<System.Tuple<System.Tuple<float, float>, LbpArea[]>>(new $Universe.alive@275-2(), array2);
    this.x.Population = array3;
    System.Tuple<System.DateTime, FSharpOption<double>> tuple = this.x.printProgress<float, LbpArea[]>(this.lastTime, this.ewma, this.generation, array3);
    System.DateTime item = tuple.Item1;
    FSharpOption<double> item2 = tuple.Item2;
    if (this.pleaseStop.WaitOne(0))
    {
        return array3;
    }
    Types.Genome[] func = this.x.breedPopulation(array3);
    return $Universe.simulate@265-1(this.x, this.pleaseStop, this.generation + 1, item, item2).Invoke(func);
}

new呼叫的IL中,找不到tail.操作。另一方面,Invoke读取的最后几行的IL

IL_00d3: call class [FSharp.Core]Microsoft.FSharp.Core.FSharpFunc`2<class BioID.GeneticLbp.Types/Genome[], class [mscorlib]System.Tuple`2<class [mscorlib]System.Tuple`2<float32, float32>, valuetype [BioID.Operations.Biometrics]BioID.Operations.Biometrics.LbpArea[]>[]> '<StartupCode$BioID-GeneticLbp>.$Universe'::'simulate@265-1'(class BioID.GeneticLbp.Universe, class [mscorlib]System.Threading.ManualResetEvent, int32, valuetype [mscorlib]System.DateTime, class [FSharp.Core]Microsoft.FSharp.Core.FSharpOption`1<float64>)
IL_00d8: ldloc.s 7
IL_00da: tail.
IL_00dc: callvirt instance !1 class [FSharp.Core]Microsoft.FSharp.Core.FSharpFunc`2<class BioID.GeneticLbp.Types/Genome[], class [mscorlib]System.Tuple`2<class [mscorlib]System.Tuple`2<float32, float32>, valuetype [BioID.Operations.Biometrics]BioID.Operations.Biometrics.LbpArea[]>[]>::Invoke(!0)
IL_00e1: ret

我不知道该怎么做。

没有|>运算符

另一个版本确实有很大的不同。从...开始

internal static System.Tuple<System.Tuple<float, float>, LbpArea[]>[] simulate@264(Universe x, System.Threading.ManualResetEvent pleaseStop, Unit unitVar0)
{
    FSharpFunc<int, FSharpFunc<System.DateTime, FSharpFunc<FSharpOption<double>, FSharpFunc<Types.Genome[], System.Tuple<System.Tuple<float, float>, LbpArea[]>[]>>>> fSharpFunc = new $Universe.simulate@265-2(x, pleaseStop);
    (($Universe.simulate@265-2)fSharpFunc).x = x;
    (($Universe.simulate@265-2)fSharpFunc).pleaseStop = pleaseStop;
    System.Tuple<System.Tuple<float, float>, LbpArea[]>[] population = x.Population;
    Types.Genome[] func;
    if (population != null && population.Length == 0)
    {
        func = x.lengthRandomlyIncreasing([email protected]@);
        return FSharpFunc<int, System.DateTime>.InvokeFast<FSharpOption<double>, FSharpFunc<Types.Genome[], System.Tuple<System.Tuple<float, float>, LbpArea[]>[]>>(fSharpFunc, 0, System.DateTime.Now, null).Invoke(func);
    }
    FSharpFunc<LbpArea[], Types.Genome> arrayToGenome = x.arrayToGenome;
    func = ArrayModule.Parallel.Map<System.Tuple<System.Tuple<float, float>, LbpArea[]>, Types.Genome>(new $Universe.simulate@296-3(arrayToGenome), population);
    return FSharpFunc<int, System.DateTime>.InvokeFast<FSharpOption<double>, FSharpFunc<Types.Genome[], System.Tuple<System.Tuple<float, float>, LbpArea[]>[]>>(fSharpFunc, 0, System.DateTime.Now, null).Invoke(func);
}

它去

// internal class simulate@265-2
public override System.Tuple<System.Tuple<float, float>, LbpArea[]>[] Invoke(int generation, System.DateTime lastTime, FSharpOption<double> ewma, Types.Genome[] population)
{
    return $Universe.simulate@265-1(this.x, this.pleaseStop, generation, lastTime, ewma, population);
}

最后

internal static System.Tuple<System.Tuple<float, float>, LbpArea[]>[] simulate@265-1(Universe x, System.Threading.ManualResetEvent pleaseStop, int generation, System.DateTime lastTime, FSharpOption<double> ewma, Types.Genome[] population)
{
    while (true)
    {
        // Playing evolution...
        if (pleaseStop.WaitOne(0))
        {
            return array3;
        }
        // Setting up parameters for next loop...
    }
    throw new System.ArgumentNullException("array");
}

tl; dr

因此可以肯定,管道操作员的使用极大地改变了程序流程。我的猜测是,最终导致异常的是两个函数之间的来回交互。

我已经阅读了F#中的Tail Calls,但是我认为它不适用于这种情况,因为我没有使用一流的函数返回单位作为值(在我的F#代码中)。

因此问题仍然存在:为什么管道运营商会在这里产生这种破坏性影响?我如何事先知道/需要注意什么?


更新2:

您可以在GitHub上找到该示例的简化版本请亲自了解inline操作员会|>更改生产IL,这不是我期望的。

在简化示例的过程中,幸运的是,我能够找到异常的真正来源。您可以检查分支以获取更多的最小变体。毕竟,它没有任何与该管,但我还是不明白这一点,因为恕我直言,还有就是尾递归。

但我最初的问题仍然存在。我只是增加一个更多:)

盖伊·编码

根据提供的最小情况,如果代码以64位版本在释放模式下运行,它将失败并产生堆栈溢出。如果代码以32位模式在发布模式下运行,则它将成功。

注意:下图显示了在32位和64位之间进行选择的选项Prefer 32-bit

增加堆栈大小将导致代码在64位释放模式下成功。这是通过使用Thread构造函数完成的

[<EntryPoint>]
let main _ =

    let test () =
        let r = KissRandom()
        let n = r.Normal()
        Seq.item 20000 n |> printfn "%f"

    /// The greatest maximum-stack-size that should be used
    /// with the 'runWithStackFrame' function.
    let STACK_LIMIT = 16777216

    /// Run a function with a custom maximum stack size.
    /// This is necessary for some functions to execute
    /// without raising a StackOverflowException.
    let runWithCustomStackSize maxStackSize fn =
        // Preconditions
        if maxStackSize < 1048576 then
            invalidArg "stackSize" "Functions should not be executed with a \
                maximum stack size of less than 1048576 bytes (1MB)."
        elif maxStackSize > STACK_LIMIT then
            invalidArg "stackSize" "The maximum size of the stack frame should \
                not exceed 16777216 bytes (16MB)."

        /// Holds the return value of the function.
        let result = ref Unchecked.defaultof<'T>

        // Create a thread with the specified maximum stack size,
        // then immediately execute the function on it.
        let thread = System.Threading.Thread ((fun () -> result := fn()), maxStackSize)
        thread.Start ()

        // Wait for the function/thread to finish and return the result.
        thread.Join ()
        !result

    /// Runs a function within a thread which has an enlarged maximum-stack-size.
    let inline runWithEnlargedStack fn =
        runWithCustomStackSize STACK_LIMIT fn


//    test ()       // Fails with stack overflow in 64-bit mode, Release
                    // Runs successfully in 32-bit mode, Release

    runWithEnlargedStack test
    
    printf "Press any key to exit: "
    System.Console.ReadKey() |> ignore
    printfn ""

    0

这段代码来自FSharp-logic-examples,尤其是Anh-Dung Phan

虽然我没有检查根本原因,但我怀疑这是因为64位项目的大小大于32位项目的大小,即使放入堆栈和两种版本的堆栈大小均保持不变,项目大小的增加使堆栈所需的内存超过1 MB的限制。

TL; DR

这是一个有趣且启发性的问题。我很高兴被问到。

最初,该问题似乎与使用|>和TCO有关,由于仍然很有价值,我将其留给了答案。我还要感谢OP的响应和帮助,很高兴能帮助与您合作而不是反对您的人。

在以下递归代码中,并|>在Visual Studio中以“调试”模式运行时,将导致StackOverflow。

如果从bin\release目录中的命令行启动它,则不会导致StackOverflow。

使用Visual Studio 15社区

[<EntryPoint>]
let main argv = 

    let largeList = 
        printfn "Creating large list"
        [
            for i in 1 .. 100000000 do
                yield i
        ]

    // causes StackOverflow in Debug
    // No StackOverflow in Release
    let sum4 l =
        printfn "testing sum4"
        let rec sumInner4 l acc =
            match l with
            | h::t -> 
                let acc = acc + h
                acc |> sumInner4 t
            | [] -> acc
        sumInner4 l 0

    let result4 = sum4 largeList
    printfn "result4: %A" result4

在Visual Studio工具栏中设置发布或调试的位置

在此处输入图片说明

调试模式下的项目选项为

在此处输入图片说明

在“发布”模式下项目的选项是

在此处输入图片说明

tldr;

In the process of testing this I created 16 different test and built them in both debug and release mode and verified if they ran to completion or threw a stack overflow. The 16 are broken down into a set of 4 with 4 cases each. The cases 1,5,9,13 are a negative and produce a stack overflow to ensure that a stack overflow can be created. Cases 2,6,10,14 are a positive to show that the tail call is working and not causing a stack overflow. Cases 3,7,11,15 show a tail call with an operation done in the same statement as the tail call, and to be one factorization away from the test cases using |>; these work as expected. Cases 4,8,12,16 use |> and shows when it does and does not work in debug mode, which is probably a surprise to many. Cases 1-4 and 9-12 use a function of the form f x y, cases 8-11 use a function of the form f x and cases 12-16 use a function of the form f x y z. I originally did the first 8 test cases but after Keith's comment did 4 more which don't use a list but still use a function of the from f x y and present the unexpected result and then did 4 more that use a function of the form f x y z.

To run a test you will have to comment out all but the one test you plan to run and the build it once in debug mode, which can then be run from within Visual Studio, and then again build it in release mode and run it. I run it from a command line to ensure I am running the release version.

[<EntryPoint>]
let main argv = 

    let largeList = 
        printfn "Creating large list"
        [
            for i in 1 .. 100000000 do
                yield i
        ]

    // causes StackOverflow in Debug
    // causes StackOverflow in Release
    //   Negative confirmation
    //   A supposed tail call that DOES cause a stack overflow in both debug and release mode
    //   options: f x y
    let sum1 l = 
        printfn "test 01: "
        let rec sum1Inner l acc =
            match l with
            | h::t -> 
                let acc = acc + h
                1 + sum1Inner t acc
            | [] -> acc
        sum1Inner l 0
        
    // No StackOverflow in Debug
    // No StackOverflow in Release
    //   Positive confirmation
    //   A tail call that DOES NOT cause a stack overflow in both debug and release mode
    //   options: f x y
    let sum2 l =
        printfn "test 02: "
        let rec sum2Inner l acc =
            match l with
            | h::t -> 
                let acc = acc + h
                sum2Inner t acc
            | [] -> acc
        sum2Inner l 0
        
    // No StackOverflow in Debug
    // No StackOverflow in Release
    //   A test case
    //   options: f x y and no |>
    let sum3 l =
        printfn "test 03: "
        let rec sum3Inner l acc =
            match l with
            | h::t -> 
                sum3Inner t (acc + h)
            | [] -> acc
        sum3Inner l 0
        
    // causes StackOverflow in Debug
    // No StackOverflow in Release
    //   A test case
    //   options: f x y and |>
    let sum4 l =
        printfn "test 04: "
        let rec sum4Inner l acc =
            match l with
            | h::t -> 
                let acc = acc + h
                acc |> sum4Inner t
            | [] -> acc
        sum4Inner l 0
        
    // causes StackOverflow in Debug
    // causes StackOverflow in Release
    //   Negative confirmation
    //   A supposed tail call that DOES cause a stack overflow in both debug and release mode
    //   options: f x
    let sum5 () =
        printfn "test 05: "
        let rec sum5Inner x =
            match x with 
            | 10000000 -> x
            | _ -> 
                let acc = x + 1
                1 + sum5Inner acc
        sum5Inner 0
        
    // No StackOverflow in Debug
    // No StackOverflow in Release
    //   Positive confirmation
    //   A tail call that DOES NOT cause a stack overflow in both debug and release mode
    //   options: f x
    let sum6 () =
        printfn "test 06: "
        let rec sum6Inner x =
            match x with 
            | 10000000 -> x
            | _ -> 
                let acc = x + 1
                sum6Inner acc
        sum6Inner 0
        
    // No StackOverflow in Debug
    // No StackOverflow in Release
    //  A test case
    //  options: f x and no |>
    let sum7 l =
        printfn "test 07: "
        let rec sum7Inner x =
            match x with 
            | 10000000 -> x
            | _ -> sum7Inner (x + 1)
        sum7Inner 0
        
    // No StackOverflow in Debug
    // No StackOverflow in Release
    //   A test case
    //   options: f x and |>
    let sum8 () =
        printfn "test 07: "
        let rec sumInner8 x =
            match x with
            | 10000000 -> x
            | _ -> 
                let acc = x + 1
                acc |> sumInner8 
        sumInner8 0

    // causes StackOverflow in Debug
    // causes StackOverflow in Release
    //   Negative confirmation"
    //   A supposed tail call that DOES cause a stack overflow in both debug and release mode"
    //   options: f x y"
    let sum9 () = 
        printfn "test 09: "
        let rec sum9Inner x y =
            match y with
            | 10000000 -> y
            | _ -> 
                let acc = x + y
                1 + sum9Inner x acc
        sum9Inner 1 0   
        
    // No StackOverflow in Debug
    // No StackOverflow in Release
    //   Positive confirmation
    //   A tail call that DOES NOT cause a stack overflow in both debug and release mode
    //   options: f x y
    let sum10 () =
        printfn "test 10: "
        let rec sum10Inner x y =
            match y with
            | 10000000 -> y
            | _ -> 
                let acc = x + y
                sum10Inner x acc
        sum10Inner 1 0

    // No StackOverflow in Debug
    // No StackOverflow in Release
    //   A test case
    //   options: f x y and no |>
    let sum11 () =
        printfn "test 11: "
        let rec sum11Inner x y =
            match y with
            | 10000000 -> y
            | _ -> 
                sum11Inner x (x + y) 
        sum11Inner 1 0
        
    // causes StackOverflow in Debug
    // No StackOverflow in Release
    //   A test case
    //   options: f x y and |>
    let sum12 () =
        printfn "test 12: "
        let rec sum12Inner x y =
            match y with
            | 10000000 -> y
            | _ -> 
                let acc = x + y
                acc |> sum12Inner x
        sum12Inner 1 0

    // causes StackOverflow in Debug
    // No StackOverflow in Release
    //   A test case"
    //   options: f x y and |>"
    let sum12 () =
        printfn "test 12: "
        let rec sum12Inner x y =
            match y with
            | 10000000 -> y
            | _ -> 
                let acc = x + y
                acc |> sum12Inner x
        sum12Inner 1 0

    // causes StackOverflow in Debug
    // causes StackOverflow in Release
    //   Negative confirmation"
    //   A supposed tail call that DOES cause a stack overflow in both debug and release mode"
    //   options: f x y"
    let sum13 () = 
        printfn "test 13: "
        let rec sum13Inner x z y =
            match y with
            | 10000000 -> y
            | _ -> 
                let acc = x + y
                1 + sum13Inner x z acc 
        sum13Inner 1 "z" 0
        
    // No StackOverflow in Debug
    // No StackOverflow in Release
    //   Positive confirmation"
    //   A tail call that DOES NOT cause a stack overflow in both debug and release mode"
    //   options: f x y"
    let sum14 () =
        printfn "test 14: "
        let rec sum14Inner x z y =
            match y with
            | 10000000 -> y
            | _ -> 
                let acc = x + y
                sum14Inner x z acc
        sum14Inner 1 "z" 0

    // No StackOverflow in Debug
    // No StackOverflow in Release
    //   A test case"
    //   options: f x y and no |>"
    let sum15 () =
        printfn "test 15: "
        let rec sum15Inner x z y =
            match y with
            | 10000000 -> y
            | _ -> 
                sum15Inner x z (x + y) 
        sum15Inner 1 "z" 0

    // causes StackOverflow in Debug
    // No StackOverflow in Release
    //   A test case"
    //   options: f x y and |>"
    let sum16 () =
        printfn "test 16: "
        let rec sum16Inner x z y =
            match y with
            | 10000000 -> y
            | _ -> 
                let acc = x + y
                acc |> sum16Inner x z
        sum16Inner 1 "z" 0

    let result1 = sum1 largeList
    printfn "result1: %A" result1

    let result2 = sum2 largeList
    printfn "result2: %A" result2

    let result3 = sum3 largeList
    printfn "result3: %A" result3

    let result4 = sum4 largeList
    printfn "result4: %A" result4

    let result5 = sum5 ()
    printfn "result5: %A" result5

    let result6 = sum6 ()
    printfn "result6: %A" result6

    let result7 = sum7 ()
    printfn "result7: %A" result7

    let result8 = sum8 ()
    printfn "result8: %A" result8

    let result9 = sum9 ()
    printfn "result9: %A" result9

    let result10 = sum10 ()
    printfn "result10: %A" result10

    let result11 = sum11 ()
    printfn "result11: %A" result11

    let result12 = sum12 ()
    printfn "result12: %A" result12

    let result13 = sum13 ()
    printfn "result13: %A" result13

    let result14 = sum14 ()
    printfn "result14: %A" result14

    let result15 = sum15 ()
    printfn "result15: %A" result15

    let result16 = sum16 ()
    printfn "result16: %A" result16
    
    printf "Press any key to exit: "
    System.Console.ReadKey() |> ignore
    printfn ""

    0 // return an integer exit code

Additional, new info

EDIT: This thread on Github has Don Syme, creator of F#, specifically mention that:

[...]其次,您是正确的,即使尾叫,我们也不保证优化对f <| x使用x |> f任何类似的功能f x

本文收集自互联网,转载请注明来源。

如有侵权,请联系 [email protected] 删除。

编辑于
0

我来说两句

0 条评论
登录 后参与评论

相关文章

函数调用后是否可以使用管道运算符调用返回?

原子既可以是谓词又可以是运算符?

JVM是否阻止尾部调用优化?

是否可以在管道内使用三元运算符?

是否有管道运算符可以在订阅解析之前运行代码?例如:http通话

我可以在gcs中向大型查询运算符气流添加可调用的python吗

是否有任何可变集合的运算符方法可以将结果写回可变运算符调用程序或参数?

是否可以向Rx.Net超时运算符添加自定义消息

这个异步管道运算符可以吗

我可以将管道运算符用作OR语句吗?

RxJs:可以将运算符作为参数传播到管道运算符中吗

是否可以将三元运算符放在函数调用中?

我可以检测是否使用错误控制运算符(@)调用了函数吗?

是否可以调用显式指定模板参数的模板化强制转换运算符?

如何重载下标运算符以返回可以是左值的可选内容?

std :: set :: insert()可以调用赋值运算符吗?

是否可以使用运算符映射自动生成指定的运算符?

如果仅实现了运算符<,是否可以使用运算符==?

是否可以模拟域的“包含”运算符?(“ in”的倒数)

是否可以从字符串创建比较运算符?

是否可以在C ++中重载运算符“ ...”?

是否可以在 Rascal 中定义自己的运算符?

是否可以重载本机数据类型的运算符?

是否可以在SQLite中将IN运算符用于夫妇

是否可以使用“?” 范围内的运算符?

是否可以编写单向三元运算符?

是否可以在python中遍历运算符(大于/小于)?

是否可以为SqlParameter实现扩展运算符?

是否可以构建响应 [] 运算符的对象