Encoder capabilities

[Delta]

Hey folks,

this is something that has been talked about a few times in the Discord, but I believe on the forum it may be a little more likely to reach the eyes of those who matter:

As we all know, there are basically two ways to use encoders in nJoy32: Either you output button pulses, or you use them to trim an axis. Both of those have their benefits and are, at least as far as I would say, implemented rather well; however, there are limitations inherent to these modes. Button pulses have a very slow reaction to quick turns, leading to delayed responses, whereas axis trim is limited by the amount of steps per revolution that the encoders have. If you use the trim function to emulate an axis, you either have a large multiplier, which means a coarse axis with large jumps, or you have an axis that takes a lot of turns to respond.

In a cockpit, you may run into controls where neither of them are adequate. Things like autopilot controls have knobs that you may turn into the same direction indefinitely (so axes are out), but you may also want to get, say, course changes in rather quickly (so button pulses are less than ideal). But of course, the standard HID game controller APIs available on Windows smack you face-first into a limitation: buttons and positional axes are pretty much the only thing most games are designed to comprehend, and sadly that includes a whole lot of flight sims.

Fortunately those sims tend to be extensible by plugins. And while (unlike some other people) I am not going to plug commercial third-party applications like S.N or A&O, there is potential to use such interfaces to get a lot more value out of modules like the FSM. Also consider that STECS can come with up to 5 encoders in its max configuration. And while I personally think that that is about three too many (remember, the Gladiator line moved from two encoders on the old NXT to one on the EVO - a rotary potentiometer option on STEM and a self-centering switch with analog functionality like the Evo Sw1 on the MTG-L sound appealing), I reckon more flexibility in their use could make them more attractive.

Some of us have heard about the LED protocol documentation Alex has put out. And while that is far from an extensive API documentation, it definitely is enough of a starting point that with a bit of experimentation, you can set up a program that can control the LEDs. I have done so myself (though it is far from what I would consider publication-worthy), and a few others have done so as well. I would propose a similar interface (maybe just a different command on feature report 0x59) to read the current internal position of an encoder so that third-party applications can tap into that and make things like the FSM-GA or the STEM much more useful. Or maybe if something like that already exists, enough information to try and figure out how it works. I would think it cannot be that much overhead when we already have "external device encoders virtualization", but of course, I recognize that it is always easy to underestimate development effort.

Looking at it in another way, there is a device out there that is literally just a coaxial pair of encoders you can plug into your PC. But it integrates with a piece of popular cockpit software really well and sells for $130.

Additionally, there is one type of HID device that uses encoder-type reporting extensively: Mice. Maybe there could be a way to output mouse axes (X, Y, Wheel) from the encoders directly instead of taking the long way through axis trim and/or button pulses?

Thanks for reading and I hope you consider some of it.

Delta

TLDR: It would be nice to have a channel that outputs raw encoder data instead of button or axis pulses over a vendor-specific HID report. Would have the potential of making FSM.GA and STEM quite a bit more useful.

[Alex Oz]

First of all, there are a few points that make sense to clarify:
1 - Push-button mode of encoders, you can make the width of the pulses as small as you like, which means you can increase the speed of issuing pulses to real time. But the question is: is such a short duration of impulses worth the game? Therefore, the default value of T_Enc has a fairly large value and a low pulse output rate.
2. You can bind the encoder to the mouse wheel at the logical level (button mode)

Now, in essence, the proposal is to make an interface with access to the internal value of the encoder.
The only reason I see is to save controller axes. As for the rest, I don’t see any advantages over the axial mode.

All this can be done, and there are two ways: a standard input-report with a specific number and structure with stationary polling, and request through feature reports. I think the first way would be more correct. It might make sense to make a special switch in the configurator.
I understand that you have a desire, so we could try.

[Delta]

First of all, there are a few points that make sense to clarify:

First of all, thanks for your quick response.

1 - Push-button mode of encoders, you can make the width of the pulses as small as you like, which means you can increase the speed of issuing pulses to real time. But the question is: is such a short duration of impulses worth the game? Therefore, the default value of T_Enc has a fairly large value and a low pulse output rate.

Yes, that is precisely the delicate balance. By the time the encoder delay is low enough to no longer cause perceptible lag, the risk for missed presses has increased significantly. Over on the Discord, I have guided quite a few users to adjusting their T_Enc. Just today, one had noticed missed presses on their STECS (which apparently had its defaults a little faster than the Evo)

2. You can bind the encoder to the mouse wheel at the logical level (button mode)

I know - that one goes through button emulation. But as far as I understand USB mice, the wheel (just like the X and Y axes) is generally implemented as a relative DV input in USB HIDs, so going through the button layer (and incurring T_Enc delays through that) could be bypassed. Admittedly I have not looked at the HID descriptor of an nJoy32 device with relative mouse emulation enabled before writing this.

Now, in essence, the proposal is to make an interface with access to the internal value of the encoder.
The only reason I see is to save controller axes. As for the rest, I don’t see any advantages over the axial mode.

Yes, pretty much. An additional advantage I see, which may be more significant than axis mode depending on the application, is indefinite continuous rotation. An axis eventually reaches its minimum and maximum values. On the other hand, between using a small multiplier and binding the axes in an external program instead of the game itself, you can probably scroll a whole lot before running into any limit. I did not even consider that option before, exposing it on the game controller interface has always been quite synonymous to mapping it in the game itself to me I guess, and for that, axis mode will not do for stuff like autopilot controls.

Note: Just tried a few things with a STEM on a GNX-HID, apparently with New Opt I can trim VC axes, and the smallest step size I can set before I run into issues with direction changes is 4, which puts me at a step size of 128 on the axis. Which would allow about 256 steps before hitting the min/max of the axis, not enough to go full circle on an autopilot course dial with 1° steps, for example.

All this can be done, and there are two ways: a standard input-report with a specific number and structure with stationary polling, and request through feature reports. I think the first way would be more correct. It might make sense to make a special switch in the configurator.
I understand that you have a desire, so we could try.

Thank you for your interest in trying. Of course I would not want to rush you or anything. I would honestly be happy with either of your proposed solutions - sending it out on the USB interface alongside the regular controller input reports actually seems like a very straightforward method, I mainly considered LED-like feature reports because that is an interface for third-party applications that you already have. I would love to try it out at some point, make my own simple implementation of a readout application, maybe help present the interface to people more experienced with making user-friendly tools out of it, if you take care of the controller side and I find the time afterwards. Thanks a lot for considering my proposal.

[Alex Oz]

Ok, thanks for reply

As soon as I get the result, I will propose some kind of solution - I think there is not much work there.

PS/ internal encoder counter in the njoy32 system - ±32k steps. Since this is a bidirectional device in principle, this is unlikely to be a limitation.

Continue monitoring this topic.

[Delta]

Thanks a lot for accepting this request. No need to rush, I know there are other important things such as stability.

+/-32k steps sounds quite generous, whether that means four steps per detent or one. Much more likely to avoid running into under- or overflow than with 128 steps per detent. Speaking of which, is there defined under-/overflow behavior? I know in some industrial standards, encoders are encoded as a 16-bit number that just over- or underflows. It starts at 0, go one step down and it is at 65535 and so on.

[Delta]

Just checked a standalone STEM with the latest build of 2.17.8 (4). The encoders seem to work a lot more reliably than on 2.17.7, even with VC axes, so the workaround brainstormed before (i.e. to use the trim axis output to feed encoder values to a PC application with minimal on-board processing) seems a lot more viable - the encoder set to 1x gives no reversal artifacts and an average step size of just shy of 4, which means you could fly dozens of circles before saturating the axes for your autopilot. With 2.16.5 working equally well, I am going to assume that that was a bug in 2.17.7 that you had fixed.
While I still think that a dedicated output is the more elegant solution, it does not seem quite as vital to rely on such for a proof-of-concept tool for encoder-to-sim integration anymore.

[Alex Oz]

So what do we have:

1. Value data is transmitted via input report with ID=8. This is a service packet for transmitting various data, so analyze the second byte (13h).

2 . Need to activate the option Virtual BUS over USB [ External ]

3 firmware (for STECS) - https://1drv.ms/u/s!AiXf55zyUARXgc0C3WJ ... w?e=aCbwWP
(In the next package this will be for all devices)

Сontroller does not process transitions at the ends in any way, so 32768+1 = - 32767

Warning!
These works are in the nature of experiments and do not have any guarantees for future support.

[Delta]

Thanks a lot. That was incredibly fast. So far it seems to be looking nice, though admittedly I have not tested out the overflow yet.

I have put together a little test application that just dumps those packages out to console using HIDAPI:

Code: Select all

#include <iostream>
#include <iomanip>
#include <iostream>
#include <hidapi/hidapi.h>

int main()
{
   int device = 0;
   int targetdevice = 0;
   hid_device* hiddev = nullptr;
   hid_device_info* devs = hid_enumerate(0x231D, 0x0000);
   for (hid_device_info* curdev = devs; curdev != nullptr; curdev = curdev->next) {
      std::wcout << "(" << targetdevice << ")" << curdev->path << ": " << curdev->product_string << " (" << curdev->serial_number << ")" << std::endl;
      targetdevice++;
   }
   std::cout << "Enter device index to connect: ";
   std::cin >> targetdevice;
   for (hid_device_info* curdev = devs; curdev != nullptr; curdev = curdev->next) {
      if (device == targetdevice) {
         hiddev = hid_open_path(curdev->path);
         break;
      }
      device++;
   }
   if (hiddev == nullptr) return -1;
   uint8_t receivebuffer[64] = { 0 };
   for (;;) {
      int bytesread = hid_read(hiddev, receivebuffer, 64);
      if (bytesread < 2) continue;
      if (receivebuffer[0] != 0x08) continue;
      if (receivebuffer[1] != 0x13) continue;
      std::cout << std::setfill('0') << std::setw(2) << std::right << std::hex << (int)receivebuffer[0];
      for (int i = 1; i < bytesread; i++) {
         std::cout << " " << std::setfill('0') << std::setw(2) << std::right << std::hex << (int)receivebuffer[i];
      }
      std::cout << std::endl;
   }
}

One thing I notice is that usually, the STECS sends this message about 8 times per second, or whenever the value changes. Which seems like a very good choice. The message number always counts up, not just when there are changes but also during the periodic transmission.

Of course, I had to do it the hard way and use a STECS with ATEM and plug both a STEM and an FSM-GA into the STECS base. That gives me a total of 8 encoders, probably among the worst you could have on a single device without address conflicts or running out of buttons. Either way, one thing I noticed is that when I turned the HDG/TRK bug on the FSM into the negative direction (i.e. encoder 6), suddenly those messages started coming in much faster, as if they were always changing. That is probably a fairly wild use case, and I do not want to hog more of your time than necessary, so make of that what you will. I for one will try and check out the side effects and interactions with other encoder actions as soon as I find the time, and try to come up with a little sample application that interfaces with a flight sim as a longer-term goal. As for today, once more thanks for what you have done.

[Delta]

Little update on the issue with the FSM-GA encoder:

The encoder also fires the virtual button for "increment" without any logical input. This behavior can be reproduced both on the "normal" 2.17.8 and the new 2.17.9, in addition to the modified 2.17.8enc. This happens as soon as the FSM-GA HDG/TRK encoder is turned into the negatives. I cannot reproduce the issue on 2.17.6 or on a setup with a more conventional encoder count (e.g. Gladiator EVO R SEM THQ FSM.GA). Might be related to being the 6th encoder on the same master (or the 5th virtual one)

[Delta]

Sorry for the triple post, but I just tried the scroll encoder on 2.17.A and it works exactly as well as I had hoped. Once more, thanks a lot.

As for the encoder endlessly firing bug, I have narrowed it down a little and it seems to be the sixth encoder configured under physical buttons. I have not tried it on other controllers yet, but at least on STECS it is quite reproducible with current versions.

If I had to guess a cause, I would wager that at some point, an unsigned subtraction is made instead of a signed one.I can provide a minimum viable configuration, if you would like. STECS Standard + FSM.GA would be the most sensible one, but if you want something that requires less gear, I could also do any STECS combo or just a basic FSM-GA on GNX-HID.

[Alex Oz]

Thanx!
This is a really strange bug of the 6th encoder.
Try next ( & encoder's Scroll )

[Delta]

Working great on non-6th encoder, on 6th encoder there seems to be a strange pattern of being locked into infinite upward scroll (with scroll set to +). Does not seem to repeat perfectly, but at least one cycle was
2 clean (i.e. stable position after the scrolling has been processed)
2 infinite up
2 clean
10 infinite up.
I seem to get 2 clean every 16 detents, that seems to be a pattern. The second round of 2 clean, 10 infinite might be more inconsistent.

[Delta]

Some more data on encoder numbers now that I have played more with shifts and modals:

- "6th Encoder", for the purpose of the bug, is the 6th item in message 0x13; if we have modals or shifts, this may be a single mode / layer.
- Modals are added directly behind the physical encoder. For predictability by 3rd-party applications, it might be more useful to add modals/shifts at the end behind all physical encoders, but I understand that this may require code restructuring that is definitely not worth the effort.
- I noticed that if you have both SHIFT#1 and MSw#1 enabled, Shift 1 and Mode 2 fire the same virtual lines. The button line of MSw#1 is used for Mode 3+ instead of Mode 2+. Mode 1 of course still uses the main encoder lines. This is reflected both in message 0x13 and in the virtual buttons firing.
- Modal switches add one too many encoders to report 0x13, there appears to be an "empty" encoder between an encoder with MSw setting and the next encoder. For example if I take STECS Mini Plus and add modal switches to all 3 encoders, I get:

1. ATEM dial Mode 1
2. ATEM dial Mode 2
3. ATEM dial Mode 3
4. ATEM dial Mode 4
5. ATEM dial Mode 5
6. Always zero
7. Ring finger wheel Mode 1
8. Ring finger wheel Mode 2
9. Ring finger wheel Mode 3
10. Ring finger wheel Mode 4
11. Ring finger wheel Mode 5
12. Always zero
13. Pinky wheel Mode 1
14. Pinky wheel Mode 2
15. Pinky wheel Mode 3
16. Pinky wheel Mode 4
17. Pinky wheel Mode 5
18. Always zero