iOS Build Environment Help Center

New (?) Clang header search rules in v. 3.35

append delete rg_software

Hello! I use iOS build env to compile a static library. The previous version of the system (3.16) works just fine, but there is a seeminly breaking change in one of the subsequent updates. Please let me know what you think about it.

Here is a minimal example.
File MyDir\MyModule.cpp has #include "stdafx.h"
File MyDir\stdafx.h has #include "../stdafx.h"
File "stdafx.h" is present in the project root folder.

I compile it with build.cmd, but I think the errors I get happen from the calls like
clang MyDir\MyModule.cpp

It complains that "../stdafx.h" isn't found. So it seems that clang does not consider ".." as a parent of "MyDir". It all looks very weird to me, and I can't figure out how to fix that.

Reply RSS

Replies

append delete #1. Pierre-Marie Baty

Hello

Does by any chance one of the files contain just one line of text ?

append delete #2. rg_software

Yes, stdafx.h is simply
#include "../stdafx.h"

P.S. I can confirm that my other clang (on linux) doesn't provide this error when I issue the same command.

append delete #3. Pierre-Marie Baty

Drop an end of line at the end of that file. You can leave the second line empty or write a dummy C comment. Is it still the same ?

If it works, I'll tell you what happened and why.

append delete #4. rg_software

Hm, strange but no. I changed stdafx.h into:

// blabla
// blabla
#include "../stdafx.h"
// blabla
// blabla

Still the same. Anyway, any ideas are appreciated.

append delete #5. Pierre-Marie Baty

OK, I thought the symlink emulation layer was getting in the way, but it's not that.

I think it's rather related to the compiler flags your project uses, and especially the include search paths.

Actually, the #include "../file.h" syntax doesn't mean "include the file called "file.h" in the parent directory of this file", but rather "include the *first* file called "file.h" that you can find in the parent directory of *any of the include search paths*".

The default compiler search paths may have changed a bit between version 3.16 and the current one. I recall at least one occurence of it, when the current directory was systematically added to the compiler search paths, which was incorrect : it's up to the Xcode project configuration (or the user-supplied CPPFLAGS) to decide whether to do so.

In your situation, are you sure the compiler include search paths that your project uses are compatible with this #include statement ?

Tip: to display them, add the "-v" option to the compiler flags when you build your project.

:: @Pierre-Marie Baty added on 16 Aug ’21 · 14:58

To be 100% correct, if I'm not mistaken, it is: the #include "../file.h" syntax means "include the *first* file called "file.h" that you can find in the parent directory of *any of the include search paths in the order at which they've been defined*".

append delete #6. rg_software

Well, for the sake of this report I do exactly this:

clang++.exe MyDir\MyModule.cpp

So there are no other paths except system paths.

If I understand correctly the behavior you describe means that the system might pick some wrong version of stdafx.h from some other folder. However, in my case it can't find any.

So in my case "cannot find ../stdafx.h" means that "MyDir" is not in the list of include search paths. I was under impression that when I compile something using

clang++.exe MyDir\bla-bla.cpp, I should get MyDir in the include path.

append delete #7. Pierre-Marie Baty

You say:

“clang++.exe MyDir\bla-bla.cpp, I should get MyDir in the include path.”

Unless I'm mistaken, that's not what the standards say, and if it worked previously, it was a mistake.

There are two types of include paths, for the <...> and "..." syntaxes respectively. The compiler manages two lists. *By default, the "..." list is empty*. When you run "clang++ -v MyDir\bla-bla.cpp", without any other flags, you should get on stderr: 

%
[ self-description info, command-line flags, compiler version etc. ]
#include "..." search starts here:
#include <...> search starts here: 
[ list of system include search paths here ]
%

Unless otherwise specified, the first list is always empty. I concede that _some_ compiler drivers add the current directory in the search paths list by default, but that's not a generality, and unless I'm mistaken, is not part of any standard.

:: @Pierre-Marie Baty added on 16 Aug ’21 · 15:40

TL/DR: just add "-IMyDir" in the compiler flags (and not "-I.", because that would be the current working directory from where you invoke the compiler).

append delete #8. rg_software

Well, I will check what's going on on other platforms, but unfortunately it is not as easy as "-IMyDir". I have *many* places with #include "../blabla.h" lines. I bet there are tons of projects that de facto include parent folders, and it's weird if recent clang thinks that "../blabla.h" shouldn't be a parent of file currently compiled.

Anyway, this is of course a clang issue. I was merely thinking that in the previous versions of iOS Build Env you might have called clang separately from inside each cpp file's folder. But seems it is not the case.

append delete #9. Pierre-Marie Baty

I understand. Your project relied on an incorrect implementation of Clang header search paths, and now that the implementation has been corrected, it breaks. It's unfortunate, and I'm not sure what to advise in this situation.

FWIW, here's what a clang -v says on macOS (this is the default clang): 

%
[pmbaty@mac-pro-de-pierre-marie ~]% clang -v empty.c
Apple clang version 12.0.5 (clang-1205.0.22.11)
Target: x86_64-apple-darwin20.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx11.0.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -Werror=implicit-function-declaration -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name empty.c -mrelocation-model pic -pic-level 2 -mframe-pointer=all -fno-strict-return -fno-rounding-math -munwind-tables -target-sdk-version=11.3 -fvisibility-inlines-hidden-static-local-var -target-cpu penryn -debugger-tuning=lldb -target-linker-version 650.9 -v -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.5 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -I/usr/local/include -internal-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/local/include -internal-isystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.5/include -internal-externc-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -internal-externc-isystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include -Wno-reorder-init-list -Wno-implicit-int-float-conversion -Wno-c99-designator -Wno-final-dtor-non-final-class -Wno-extra-semi-stmt -Wno-misleading-indentation -Wno-quoted-include-in-framework-header -Wno-implicit-fallthrough -Wno-enum-enum-conversion -Wno-enum-float-conversion -Wno-elaborated-enum-base -fdebug-compilation-dir /Users/pmbaty -ferror-limit 19 -stack-protector 1 -fstack-check -mdarwin-stkchk-strong-link -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fmax-type-align=16 -fcommon -fcolor-diagnostics -clang-vendor-feature=+disableNonDependentMemberExprInCurrentInstantiation -fno-odr-hash-protocols -mllvm -disable-aligned-alloc-awareness=1 -o /var/folders/3h/8x2fp6b91p595gdvk7qsdw5w0000gn/T/empty-f6f8cd.o -x c empty.c
clang -cc1 version 12.0.5 (clang-1205.0.22.11) default target x86_64-apple-darwin20.3.0
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/local/include"
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.5/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list.
%

The user-supplied directory list is empty here too. Which makes me believe that, contrarily to what you think, not so many projects will break. But some will sure do.

I still want to make something 100% clear, when you say:

“it's weird if recent clang thinks that "../blabla.h" shouldn't be a parent of file currently compiled.”

Actually this has never been the case (see my definition of the meaning of the #include "..." syntax for the preprocessor). This is not a particularity of clang, this is how C preprocessors have always worked, because that's what the language reference says.

I encourage you to fix your code so that it's conformant to the C specification. You'll save time in the long run, it'll make it more portable, and well, should there be one last reason, don't underestimate the importance of having learned something new today. Such a reasoning has saved me many times.

:: @Pierre-Marie Baty added on 16 Aug ’21 · 17:15

Edit: I don't want to say stupid things, so I'll refresh my memories on this topic. I'll read the reference again and post back.

:: @Pierre-Marie Baty added on 16 Aug ’21 · 17:18

Here's what GCC says:

“2.3 Search Path

By default, the preprocessor looks for header files included by the quote form of the directive #include "file" first relative to the directory of the current file, and then in a preconfigured list of standard system directories. For example, if /usr/include/sys/stat.h contains #include "types.h", GCC looks for types.h first in /usr/include/sys, then in its usual search path.”

So, #include "../blabla.h" works with GCC and I was *wrong*. Either it's a design choice by the GCC authors, or this behaviour is really how all compilers are supposed to work. I'm looking now on how this is supposed to be implemented in clang...

:: @Pierre-Marie Baty added on 16 Aug ’21 · 17:23

Well, it looks like it *really* is a Clang design choice after all. Here's what the reference says: https://en.cppreference.com/w/c/preprocessor/include

“#include "filename" (2)

2) Searches for the file in implementation-defined manner. The intent of this syntax is to search for the files that are not controlled by the implementation. Typical implementations first search the directory where the current file resides and, only if the file is not found, search the standard include directories as with (1).”

Conclusion: it's implementation-dependent. And we should look in the Clang source code for confirmation. Pah...

:: @Pierre-Marie Baty added on 16 Aug ’21 · 17:37

Okay, end word. I made a test on macOS with Clang 12 (same compiler as with the builder, except the minor version). I created a source file in a subdirectory: 

% test/include-test.c
#include "../blabla.h"

int ENTRYPOINT (int argc, char **argv)
{
	printf ("well, it works.\n");
	exit (0);
}
%

And I put an include like yours in the current directory: 

% blabla.h
#include <stdio.h>
#include <stdlib.h>
#define ENTRYPOINT main
%

Results: 

%
[pmbaty@mac-pro-de-pierre-marie ~/Desktop]% clang test/include-test.c 
[pmbaty@mac-pro-de-pierre-marie ~/Desktop]% ./a.out 
well, it works.
[pmbaty@mac-pro-de-pierre-marie ~/Desktop]% 
%

So, you *were* right and I was wrong. My apologies. The search paths implementation in my Clang compiler on Windows is dissymetrical with the Clang compiler on macOS. I'll see what I can do.

append delete #10. rg_software

Thanks a lot for your experiments!
I was trying to do some tests as well and indeed it behaves in a somewhat surprising way. For example, I tried adding -IMyDir, and it didn't work too.

BTW, if we take that "implementation defined" clause at face value, it doesn't even say that when I do #include "myfile.h" in myfile.cpp, they are to be expected in the same folder!

append delete #11. Pierre-Marie Baty

I think I've found the problem. I'm recompiling another clang (it takes quite some time unfortunately...) and will report the results.

append delete #12. Pierre-Marie Baty

The fix is confirmed working. I'm tidying the patch and will publish an update soon.

:: @Pierre-Marie Baty added on 18 Aug ’21 · 11:40

FYI (while it recompiles), the problem was that when pushing include search paths on the search list, empty paths ( = current directory) were ignored, but dot-prefixes (".") were not. My POSIX symlink emulation layer would return an empty string as the path component when evaluating whether a particular path contained a symlink that should be resolved and that path was a file in the current directory. Making it return a dot "." as the path component in this case made Clang accept it as a possible search path, restoring identical functionality to the official *NIX Clang distributions (with the addition of plaintext symlinks support on Win32).

append delete #13. rg_software

Great, thanks a lot for your work.

append delete #14. Pierre-Marie Baty

Version 3.36 is out. Let me know if it fixes the problem.

append delete #15. rg_software

Yes, I confirm it works. I had to modify my scripts reading from output paths (used to be build/bla-bla.lib, now it is build/-iphoneos/bla-bla/bla-bla.lib), but otherwise no problems anymore. Thanks again!

append delete #16. Pierre-Marie Baty

Glad to hear it. I can consider this case closed. Happy building :)

Reply

(Leave this as-is, it’s a trap!)

There is no need to “register”, just enter the same name + password of your choice every time.

Pro tip: Use markup to add links, quotes and more.

Moderators: Pierre-Marie Baty