question

Kalle L. avatar image
Kalle L. asked ·

Object detection is off with Firefox 24.0 ESR

Hi,

We have an ESM script that's using Firefox 24.0, and one particular agent is having this issue: the browser object are recognized, but at the wrong location. So for example if we try to click the username field at a logon screen to activate the textbox, it will locate the object, but click way off the actual textbox, thus failing to activate it. This happens for all objects on screen.

If I use the object identifier, it will not find the objects at the correct location, but instead a bit off both from the X and Y axis. I can also navigate to a specific object via the tree, but when I click locate, the identifier will flash at the wrong place.

Has anyone seen this type of behavior before, and if so, what was the cause? I know I could/should open a ticket, but I will be out of office soon for a few weeks so I won't be able to follow up on it properly. That's why I thought I could ask for some general tips from here...

esmscripting
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

1 Answer

Yuriy L. avatar image
Yuriy L. answered ·

@Kalle L.

Hi Kalle,

Forum, of course, is a great place. Hopefully other participants will bring their ideas. And it gives a chance to discuss how the problem can be looked at, while leaving actions up to the reader. You probably already know most, of what I am going to say, but... Here is my take.

1. You say "one particular agent is having this issue". What makes this machine different? Is it possible to make it identical to other machines? Cloning usually allows to make several machines behave the same way. If this is not possible, here are couple things to try:

1.1. Disable Firefox accelerated rendering, http://forums.mozillazine.org/viewtopic.php?t=1775755.

1.2. If the machine is running Windows 10, or Windows 8.1, or Windows Server 2016, or Windows Server 2012 R2, disabling High DPI scaling might help, https://support.microsoft.com/en-us/help/2900023/some-desktop-applications-may-appear-blurred-on-high-dpi-displays and https://appuals.com/disable-display-scaling-on-high-dpi-devices-in-windows-10/.

2. The coordinates you are getting are wrong. Coordinates are needed mostly for emulating mouse functionality. Try to avoid using mouse. For instance, instead of .Click you can put .SetFocus. Using Keyboard section of Monitoring Applications with Un-recognized Controls documentation page contains some approaches you may consider.

3. You can quit using controls altogether. In addition to recommendation in the already quoted Monitoring Applications with Un-recognized Controls documentation page, you can also use Screen object introduced in 12.5.3. Unfortunately, I cannot point you to any documentation page – we are a bit behind there. But Object Browser (View->Object Browser) contains signatures and brief descriptions of its methods and properties.

If you open a support ticket, we will, of course, help you with the approaches listed above and we may also try something else.

Thank you,

Yuriy

5 comments Share
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

Hi Yuriy,

Thanks for the response! To be clear, when I said the objects are found from the wrong location, I didn't mean I was using any static X and Y coordinates in the script. What I mean is, if I define:

MozillaEditBox("Name=password").Click

...it will click on the right editbox at all the other agents. But with this one agent, the click will miss. The reason being, that particular agent simply locates all objects at the wrong place.

I suppose if I can figure out the exact X and Y "offset" for this agent, I could perhaps apply it each time I access those objects. Well, assuming that offset is the same for all objects.

Otherwise there's nothing special about this agent to my knowledge, it's a Windows 7 installation and we already have another agent in this environment with that OS up and running normally.

I will check your other suggestions as well, cheers.

0 Likes 0 · ·

@Kalle L.

Hi Kalle,

I understood you exactly the way you describe the problem. Click operations depend upon the coordinates, of course, please see https://community.dynatrace.com/community/pages/viewpage.action?pageId=237341108. By default .Click clicks at the center of the control. You can put coordinates for .Click explicitly, which would allow to click outside of the determined control rectangle. The coordinates in the .Click method are related to the upper-left corner of the control. So, if the determined rectangle is to the right and below the actual rectangle, you will need to put negative coordinates. If the determined rectangle is to the left and above the actual rectangle, you will need to put coordinates higher than the determined size of the control. Using this approach, though, would make it difficult to maintain your script portable across your different environments.

By the way, here is another potential reason for the problem with one specific environment. Possibly Firefox starts there with scaling not equal to 100%. Sending "{Ctrl 0}" (control-zero) to Firefox would ensure that the scaling factor is equal to 100%. If this is the case, putting necessary code prior to any operations with HTML controls would address the problem, and such script would be portable across your different environments.

Thank you,

Yuriy

0 Likes 0 · ·

Very helpful comments again Yuriy, thanks! The "mis-identify" is indeed in the upper left in proportion to the actual object, so applying positive coordinates would be my next test. Good point about Ctrl+0 also, that's something I'll definitely add to the script also.

0 Likes 0 · ·

HI Kalle,

Have you checked on the scaling factor?

0 Likes 0 · ·

Hi,

The scaling was already 100 %, so it wasn't about that. I was able to get it to work by applying custom coordinates for each click. I couldn't use the same values for all objects though, as the object's size affects how much I need to adjust the click. Nonetheless, it's working now even though the root cause is still a mystery to me.

0 Likes 0 · ·