Dependencies with static libraries

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Dependencies with static libraries

Stephan Menzel
Hello CMake community,

I am observing a strange problem regarding dependencies to static libs in executables. I am not entirely sure but it seems this started since I upgraded to the 3.11 line. I am using Windows and MSVC14.

Basically, my code base contains a number of static libraries that are linked together to a few executables. Very basic and down to earth. Like this:

set(MY_SERVER_SRC
ServerSource1.cpp
ServerSource2.cpp
)

set(MY_SERVER_HDR
ServerConfig.hpp
Header1.hpp
Header2.hpp
)

add_library(my_server STATIC ${MY_SERVER_SRC} ${MY_SERVER_HDR})
target_link_libraries(my_server 
some_tools
some_base
${Boost_LIBRARIES}
${OPENSSL_LIBRARIES}
)
add_executable(serverd main.cpp)
target_link_libraries(serverd 
my_server 
some_tools
some_base

${Boost_LIBRARIES}
${OPENSSL_LIBRARIES}
)

So a few big libs with the beef in them and then a small main.cpp containing executable. Nothing fancy.

What gives me trouble now is that dependencies within that seem broken. In above example, when I edit ServerSource1.cpp and then hit "Build", I see ServerSource1.o being compiled and the library my_server being linked. Like expected. Problem is, serverd does not get linked. Or at least I see no hint of it being linked (In MSVC's output window).
So, my question is: What could cause this? I there anything 3.11.x does differently which causes those dependecies to break? And how can I fix this?

This is quickly turning into a major problem for us here as it means that we cannot reliably build anymore because whenever we change something and hit build we can't be sure we get a linked executable anymore unless we do a complete rebuild, which takes a long time (In fact, I wrote this message waiting for one).

Thanks for any hint!

Stephan



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Dependencies with static libraries

Innokentiy Alaytsev
Hello!

I hope you will forgive me a silly question: did you consider checking last
change date of the executable? And is there any way you can ensure that the
changes to the library a present in the executable?

Best regards,
Innokentiy Alaytsev



--
Sent from: http://cmake.3232098.n2.nabble.com/
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Dependencies with static libraries

Stephan Menzel
Hello Innokentiy,

On Thu, Jun 28, 2018 at 4:17 PM, Innokentiy Alaytsev <[hidden email]> wrote:
Hello!

I hope you will forgive me a silly question: did you consider checking last
change date of the executable? And is there any way you can ensure that the
changes to the library a present in the executable?

So far the problem appears too weird to me and I have not tracked it down. Basically, my way of knowing the changes are not present is debugging for half a day and getting crazy because whatever I do seems to have no effect whatsoever ;-)

Since you asked it though, I just did the obvious check for the timestamp and I see the executable is being updated. It's just not mentioned in the output log. Perhaps I really am imagining all this and was misled by the missing output.

Sorry for the noise, I will double check this.

Stephan



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Dependencies with static libraries

Hendrik Sattler
In reply to this post by Stephan Menzel
Are you sure you hit the right build menu item? Maybe you are just building the current target. At least that would explain it. In MSVC you must either build the solution or the ALL_BUILD target for this to work.


Am 28. Juni 2018 16:07:33 MESZ schrieb Stephan Menzel <[hidden email]>:
Hello CMake community,

I am observing a strange problem regarding dependencies to static libs in executables. I am not entirely sure but it seems this started since I upgraded to the 3.11 line. I am using Windows and MSVC14.

Basically, my code base contains a number of static libraries that are linked together to a few executables. Very basic and down to earth. Like this:

set(MY_SERVER_SRC
ServerSource1.cpp
ServerSource2.cpp
)

set(MY_SERVER_HDR
ServerConfig.hpp
Header1.hpp
Header2.hpp
)

add_library(my_server STATIC ${MY_SERVER_SRC} ${MY_SERVER_HDR})
target_link_libraries(my_server 
some_tools
some_base
${Boost_LIBRARIES}
${OPENSSL_LIBRARIES}
)
add_executable(serverd main.cpp)
target_link_libraries(serverd 
my_server 
some_tools
some_base

${Boost_LIBRARIES}
${OPENSSL_LIBRARIES}
)

So a few big libs with the beef in them and then a small main.cpp containing executable. Nothing fancy.

What gives me trouble now is that dependencies within that seem broken. In above example, when I edit ServerSource1.cpp and then hit "Build", I see ServerSource1.o being compiled and the library my_server being linked. Like expected. Problem is, serverd does not get linked. Or at least I see no hint of it being linked (In MSVC's output window).
So, my question is: What could cause this? I there anything 3.11.x does differently which causes those dependecies to break? And how can I fix this?

This is quickly turning into a major problem for us here as it means that we cannot reliably build anymore because whenever we change something and hit build we can't be sure we get a linked executable anymore unless we do a complete rebuild, which takes a long time (In fact, I wrote this message waiting for one).

Thanks for any hint!

Stephan



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Dependencies with static libraries

Stephan Menzel
In reply to this post by Stephan Menzel
On Thu, Jun 28, 2018 at 4:07 PM Stephan Menzel <[hidden email]> wrote:
I am observing a strange problem regarding dependencies to static libs in executables. I am not entirely sure but it seems this started since I upgraded to the 3.11 line. I am using Windows and MSVC14.

Basically, my code base contains a number of static libraries that are linked together to a few executables. Very basic and down to earth. Like this:

set(MY_SERVER_SRC
ServerSource1.cpp
ServerSource2.cpp
)

set(MY_SERVER_HDR
ServerConfig.hpp
Header1.hpp
Header2.hpp
)

add_library(my_server STATIC ${MY_SERVER_SRC} ${MY_SERVER_HDR})
target_link_libraries(my_server 
some_tools
some_base
${Boost_LIBRARIES}
${OPENSSL_LIBRARIES}
)
add_executable(serverd main.cpp)
target_link_libraries(serverd 
my_server 
some_tools
some_base

${Boost_LIBRARIES}
${OPENSSL_LIBRARIES}
)

So a few big libs with the beef in them and then a small main.cpp containing executable. Nothing fancy.

What gives me trouble now is that dependencies within that seem broken. In above example, when I edit ServerSource1.cpp and then hit "Build", I see ServerSource1.o being compiled and the library my_server being linked. Like expected. Problem is, serverd does not get linked. Or at least I see no hint of it being linked (In MSVC's output window).
So, my question is: What could cause this? I there anything 3.11.x does differently which causes those dependecies to break? And how can I fix this?

This is quickly turning into a major problem for us here as it means that we cannot reliably build anymore because whenever we change something and hit build we can't be sure we get a linked executable anymore unless we do a complete rebuild, which takes a long time (In fact, I wrote this message waiting for one).


I have a little update here in case anyone experiences the same. Since I first noticed it when I posted here the problem was degrading. Back then I thought I couldn't track it down because it wasn't always happening. What led me to believe it was reproducible was the lack of any kind of output in the console.

Unfortunately since then the problem has turned from a major nuisance into a much bigger issue because right now it actually is always reproducible. It never works anymore.

The only way to bring VS to re-link my executable is to delete the old one. Any change in the lib 'my_server' leads to the lib being compiled and linked fine but the server executable never is.

I have checked this many times using timestamps. Now I usually see the lib with a 'right now' timestamp and the executable with a much older one. Several other developers here in the team experience the same thing so I don't think it is my computer or the installation or anything.
I have also tried updating to 3.12.1 and the issue is still there. 

As for reasons I am still in the dark. My money would be on some kind of race condition. When the problem gradually got worse I also occasionally saw a linker error saying the "program database is already in use". Waiting a few seconds and then clicking "Build Solution" again generally solved that but still I suppose this should never happen unless the binary is running or otherwise in use.

Also since then we gradually added rather template heavy code to that static lib which (in Debug mode) is now 250M in size. My guess is that some processes are still working on the lib when the linker tries to build the executable out of it. The linker fails, lack of console output hides that from me and with the lib growing this got more and more likely over time and is now always the case.

I know, it's a long shot. Counter evidence against this hypothesis is the fact that in Release mode the problem occurs just as much and there the data is much smaller, only about 50M, with no pdbs.

Meanwhile I have also double checked all kinds of flags or defines we use and did not find anything out of the ordinary. I have also verified the dependencies as they are shown in the visual studio project properties. They look very much like I expect them to look.

I am doubling down efforts into investigating this now as it has cost us dearly the last couple of weeks as new developers stumbled into it without knowing about it. Will post results here. Of course I also appreciate new ideas about possible reasons.

Cheers,
Stephan



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake