Can't stop low level hooks in powershell (c# hook)-CodePudding

after an evening of arguments with both chat gpt and bing chat, I now turn to the (hopefully) wiser people of stackoverflow. My problem is this: I have low level hooks implemented in c#. I subscribe to the events in a powershell class, which I then use in a GUI. Subscribing to the events and starting the hooks works just fine, however, I can't stop the hooks from my class or GUI program. Currently I'm thinking that whenever I press 'f4' the hooks should stop, but I keep getting 'cannot call a method on a null-valued expression', and I really don't understand why it would be null-valued or how I would solve it. Here comes my code, I don't think it's necessary to show the implementation of the hook or the GUI, but let me know otherwise.

class InputRecorder {
    [KeyboardHookExample.KeyboardHook] $kh
    [Ikst.MouseHook.MouseHook] $mh
    [System.Windows.Forms.ListView] $list

    InputRecorder() {
        $this.kh = New-Object KeyboardHookExample.KeyboardHook
        $this.mh = New-Object Ikst.MouseHook.MouseHook

        # Store the reference to the class instance
        $self = $this

        # Define the event handler for KeyDown
        $self.kh.add_KeyDown({
            param($sender, $e)
            $vkCode = $sender.vkCode
            Write-Host $vkCode
            if ($vkCode -eq 115) {
                $self.kh.Stop()
                $self.mh.Stop()
            }

            $charCode = [Win32.NativeMethods]::MapVirtualKey($vkCode, 2)
            $char = [char]$charCode

            Write-Host $char
        })

        # Define the event handler for LeftButtonDown
        $self.mh.add_LeftButtonDown({
            param($sender, $e)
            $mousePosition = $sender.pt
            $y = $mousePosition.y
            $x = $mousePosition.x
            $item = New-Object System.Windows.Forms.ListViewItem
            $item.ToolTipText = $global:dict["LeftClickCode"] -f $x, $y
            $item.Text = $global:dict["LeftClickDescription"] -f $x, $y
            $CMDList.Items.Add($item)
        })

        # Start the keyboard and mouse hooks
        $self.kh.Start()
        $self.mh.Start()
    }
    [System.Collections.Concurrent.ConcurrentBag[string]] getList() {
        return $this.list
    }

    [void] StopHooks() {
        $this.mh.Stop()
        $this.kh.Stop()
    }
}

CodePudding user response：

The problem is that the method-local $self variable isn't available from inside the script block serving as an event delegate passed to the .add_KeyDown() method.

While it is understandable to expect PowerShell's dynamic scoping to also apply inside PowerShell's custom classes, that is not the case:

A script block passed as an event delegate to a .NET method from inside a class:

does not see the local variables of the enclosing method - unless you explicitly capture them by calling .GetNewClosure() on the script block.
- What it does see in the absence of .GetNewClosure() are the variables from the class-defining scope, i.e. those defined outside the class, in the scope in which the class is defined (and from that scope's ancestral scopes), because it runs in a grandchild scope of that scope.^[1]
does not see $this as referring to the instance of the enclosing class, because - unfortunately - the automatic $this variable in event delegates shadows the class-level definition of $this and instead refers to the event sender.
- And because class-internal use of $this is the only way to gain access to instance variables (properties) of a class, the latter are shadowed too.

There are two solution options:

On a per-method basis (to make your attempt work):
- Define a method-local variable such as $self and then call .GetNewClosure() on every event-delegate script block passed to .NET methods from the same method, which allows you to access the method-local variables in that script block.
- However, note that .GetNewClosure() will make your script block lose access to the variables from the class-defining scope.
Preferably, at the class level (no method-specific code needed):
- Use (Get-Variable -Scope 1 -ValueOnly this) to gain access to the the class instance at hand, i.e. to the shadowed version of $this
- This obviates the need for method-specific logic and also preserves access to variables from the class-defining scope (if needed).

The following self-contained sample code demonstrates both approaches:

For easy visualization, a WinForms form is created (the scoping issues apply equally), with two buttons whose event handlers demonstrate either approach above.
Clicking either button updates the form's title text with distinct text, and the ability to do so implies that a reference to the enclosing class instance was successfully obtained from inside the event-handler script blocks.

Important:

Be sure to execute the following first, before invoking the code below via a script file:
```
Add-Type -ErrorAction Stop -AssemblyName System.Windows.Forms
```
This is - unfortunately - necessary, because (up to at least PowerShell 7.4.0) any .NET types referenced in a class definition must have been loaded into the session before the script is parsed (loaded) - see this answer for background information.

# IMPORTANT: 
#  * Run 
#      Add-Type -AssemblyName System.Windows.Forms
#    BEFORE invoking this script.

using namespace System.Windows.Forms
using namespace System.Drawing

class FormWrapper {
  [Form] $form

  FormWrapper() {
    # Create a form with two buttons that update the form's caption (title text).
    $this.form = [Form] @{ Text = "Sample"; Size = [Size]::new(360, 90); StartPosition = 'CenterScreen' }
    $this.form.Controls.AddRange(@(
        [Button] @{
          Name = 'button1'
          Location = [Point]::new(30, 10); Size = [Size]::new(130, 30)
          Text = "With Get-Variable"
        }
        [Button] @{
          Name = 'button2'
          Location = [Point]::new(190, 10); Size = [Size]::new(130, 30)
          Text = "With GetNewClosure"
        }
      ))

    # Get-Variable approach.
    $this.form.Controls['button1'].add_Click({
        param($sender, [System.EventArgs] $evtArgs)
        # Obtain the shadowed $this variable value to refer to
        # this class instance and access its $form instance variable (property)
        (Get-Variable -Scope 1 -ValueOnly this).form.Text = 'Button clicked (Get-Variable)'
      })

    # Local variable   .GetNewClosure() approach.
    # Define a method-local $self variable to cache the value of $this.
    $self = $this
    # !! YOU MUST CALL .GetNewClosure() on the event-delegate script
    # !! block to capture the local $self variable
    $this.form.Controls['button2'].add_Click({
        param($sender, [System.EventArgs] $evtArgs)
        # Use the captured local $self variable to refer to this class instance.
        $self.form.Text = 'Button clicked (.GetNewClosure)'
      }.GetNewClosure())
  
  }

  ShowDialog() {
    # Display the dialog modally
    $null = $this.form.ShowDialog()
  }

}

# Instantiate the class and show the form.
[FormWrapper]::new().ShowDialog()

^{[1] The immediate parent scope is the class instance at hand, but (a) there appears to be no method-level scope (hence no access to method-local variables) and (b) at the instance level the only variable that is defined is $this, which is shadowed, as explained later (and any instance variables (properties) must be accessed via $this.)}