How remove this optimization of hexrays?

IDA General
1

I think hexrays forgot what it is not
and it is not a compiler
and it should not optimize the code that the compiler left

so how to disable/drop/remove such optimization?

  1. hexrays looks for similar patterns in the microcode and replaces them with one,
    adding or redirecting via GOTO to them

if (a()) { return b(); return; }
if (c()) { return b(); return; }
if (d()) { return b(); return; }
if (e()) { return b(); return; }

is replaced by

if (a()) { A: return b(); return; }
if (c()) { goto A; }
if (d()) { goto A; }
if (e()) { goto A; }

or even worse
is replaced by

if (a() || c() || d() || e()) { return b(); return; }

why is it bad?

firstly, the original instructions are deleted
if the compiler left them, it means he considered it correct
or he did not optimize it and you can see the original idea of ​​the developer himself
and hexrays has no right to put itself above the compiler and optimize something

secondly, with the advent of eh34, this will interfere with creating correct places for installing try/catch blocks

  1. replaces all comparisons with zero with regular uint a; if(a>0) → if(a)
    you will say yes, this comparison does not make sense
    I will say, and what right do you have to decide for the compiler?
    again, if the compiler left it - then I want to see it

moreover, in some platforms, these can be different types, signed or unsigned
and this code was compiled in the unsigned version

that’s why I really want to see the comparison with zero

  1. replaces if (a == 0 || a == 1) → if (a < 2)
    which subsequently creates problems with using enum
    instead of enum { A = 0, B, C }; if (a == A || a == B) → if (a < C)

there are actually a lot of such optimizations
I can’t remember absolutely all of them now
but I can add them to this topic as I come across or remember them

if you don’t want to remove these optimizations, then is it possible to add something to the API so that you can remove these optimizations by writing your own plugin

1 Like
2

If the decompiler didn’t perform optimizations because “the compiler knows better”, you’d be looking at unreadable mess with all individual operations, gotos and dead code left as-is. We have tried to pick optimizations and transformations which produce compact but (usually) correct representation of the underlying binary code. Some of them can be changed in decompiler options or config file, others currently not.

Additionally, It’s a little difficult to discuss abstract code snippets without the underlying code. If you are certain that the decompiler made a wrong decision in a specific situation, we’d encourage you to submit reports with binaries (e.g. via the support portal, Help > Send Database, or by email) so that they can be investigated in detail.

2 Likes
3 
#include <stdio.h>

#ifdef _WIN32
typedef unsigned long MYTIME;
#else
typedef long MYTIME;
#endif
#ifdef _WIN32
#define __PRETTY_FUNCTION__ __FUNCSIG__
#endif

MYTIME get_my_time()
{
	printf("%s %d\n", __PRETTY_FUNCTION__, __LINE__);
	return 10;
}

void check_time()
{
	MYTIME last_time = get_my_time();
	while (1)
	{
		MYTIME cur_time = get_my_time();
		MYTIME interval = cur_time - last_time;
		if (interval < 0 || interval > 10)
		{
			printf("%s %d\n", __PRETTY_FUNCTION__, __LINE__);
			return;
		}
		printf("%s %d\n", __PRETTY_FUNCTION__, __LINE__);
	}
}

unsigned int get_num()
{
	printf("%s %d\n", __PRETTY_FUNCTION__, __LINE__);
	return 10;
}

void check_zero()
{
	if (get_num() > 0)
	{
		printf("%s %d\n", __PRETTY_FUNCTION__, __LINE__);
	} else
	{
		printf("%s %d\n", __PRETTY_FUNCTION__, __LINE__);
	}
}

enum TEST
{
	TEST_0 = 0,
	TEST_1 = 1,
	TEST_2 = 2,
	TEST_3 = 3,
};


struct NESTED_RETURN
{
	int test = 0;
	TEST t = TEST_0;

	void check_enum1()
	{
		if (t == TEST_0 || t == TEST_1)
		{
			printf("%s %d\n", __PRETTY_FUNCTION__, __LINE__);
		} else
		{
			printf("%s %d\n", __PRETTY_FUNCTION__, __LINE__);
		}
	}

	void check_enum2()
	{
		if (t == TEST_0 || t == TEST_1 || t == TEST_2 || t == TEST_3)
		{
			printf("%s %d\n", __PRETTY_FUNCTION__, __LINE__);
		} else
		{
			printf("%s %d\n", __PRETTY_FUNCTION__, __LINE__);
		}
	}
};


void check_return()
{
	NESTED_RETURN nr;
	nr.check_enum1();
	nr.check_enum2();
}

int main()
{
	check_time();
	check_zero();
	check_return();
	return 0;
}

compile under windows without optimization
i hope there is no need the binary file
hexrays output

void check_time(void)
{
  unsigned int last_time; // [rsp+24h] [rbp-14h]

  last_time = get_my_time();
  while ( get_my_time() - last_time <= 10 )
    printf_0("%s %d\n", "void __cdecl check_time(void)", 29);
  printf_0("%s %d\n", "void __cdecl check_time(void)", 26);
}

void check_zero(void)
{
  if ( get_num() )
    printf_0("%s %d\n", "void __cdecl check_zero(void)", 43);
  else
    printf_0("%s %d\n", "void __cdecl check_zero(void)", 46);
}

void __fastcall NESTED_RETURN::check_enum1(NESTED_RETURN *this)
{
  if ( this->t > (unsigned int)TEST_1 )
    printf_0("%s %d\n", "void __cdecl NESTED_RETURN::check_enum1(void)", 71);
  else
    printf_0("%s %d\n", "void __cdecl NESTED_RETURN::check_enum1(void)", 68);
}

void __fastcall NESTED_RETURN::check_enum2(NESTED_RETURN *this)
{
  if ( this->t >= 4u )
    printf_0("%s %d\n", "void __cdecl NESTED_RETURN::check_enum2(void)", 82);
  else
    printf_0("%s %d\n", "void __cdecl NESTED_RETURN::check_enum2(void)", 79);
}

in assembler all instruction are present
hexrays just remove it

may be your add into hexrays some optimization level option like compiler O0 O2 O3
to disable some optimization by hexrays

1 Like
4

Adding optimization levels is not in our plans.

Your example is quite long, could you tell me what instructions were omitted to change the code semantics?

1 Like
5

assembler

.text:0000000140006FB9 028                 mov     rax, [rsp+28h+this]
.text:0000000140006FBE 028                 cmp     dword ptr [rax+4], 0
.text:0000000140006FC2 028                 jz      short loc_140006FE5
.text:0000000140006FC4 028                 mov     rax, [rsp+28h+this]
.text:0000000140006FC9 028                 cmp     dword ptr [rax+4], 1
.text:0000000140006FCD 028                 jz      short loc_140006FE5
.text:0000000140006FCF 028                 mov     rax, [rsp+28h+this]
.text:0000000140006FD4 028                 cmp     dword ptr [rax+4], 2
.text:0000000140006FD8 028                 jz      short loc_140006FE5
.text:0000000140006FDA 028                 mov     rax, [rsp+28h+this]
.text:0000000140006FDF 028                 cmp     dword ptr [rax+4], 3
.text:0000000140006FE3 028                 jnz     short loc_140007000

decompiler

  if ( this->t >= 4u )
1 Like
6

Aren’t they equivalent in this context?

1 Like
7

topic not about equivalent
topic about remove/drop/disable some optimization
let user decide about it
not the hexrays

and again
hexrays not a compiler to optimize this comparison

1 Like
8

As I said, we do not plan to introduce these options.

3 Likes
9

Any reason why not? Because it seems like really nice thing to have…

1 Like
10

such and similar optimizations
it is better to split into another plugin product for hexray
which will obviously be useful for all kinds of antivirus companies
that do not deal with code recovery as such
and such optimizations will not interfere with them

1 Like