Btukfyl

I was thinking why are there (in all programming languages I have learned, such as C++, Java, Python) standard libraries like stdlib, instead of having similar "functions" being a primitive of the language itself.

Allow me to expand somewhat on @Vincent's (+1) good answer:

Why couldn't the compiler simply translate a function call into a set of instructions?

It can, and does so via at least two mechanisms:

• inlining a function call — during translation, the compiler can replace a source code call with its implementation directly inline instead of making an actual call to the function.  Still the function needs to have an implementation defined somewhere and that can be in the standard library.




• intrinsic function — intrinsics are functions that the compiler has been informed of without necessarily finding the function in a library.  These are usually reserved for hardware features that are not practically accessible in any other way, being so simple that even the overhead of a call to assembly language library function is considered high.  (The compiler can generally only automatically inline source code in its language, but not assembly functions, which is where intrinsic mechanism comes in.)

Still these being said, the best option sometimes is for the compiler to translate a function call in the source language into a function call in the machine code.  Recursion, virtual methods, and sheer size are some reasons that inlining is not always possible/practical.  (Another reason is intent of the build, such as separate compilation (object modules), separate load units (e.g. DLLs)).

There's no real advantage to making most standard library functions intrisics either (that would hard code a lot more knowledge into the compiler for no real advantage), so a machine code call again is often most appropriate.

C is a notable language that arguably omitted other explicit language statements in favor standard library functions.  Though libraries pre-existed, this language made a shift to doing more work from standard library functions and less as explicit statements in the grammar of the language.  IO in other languages, for example, was frequently given its own syntax in the form of various statements, whereas the C grammar does not define any IO statements, simply instead deferring to its standard library to provide that, all accessible via function calls, which the compiler already knows how to do.

This is simply to keep the language itself as simple as possible. You need to distinguish between a feature of the language, such as a type of loop or ways to pass parameters to functions and so on, and common functionality that most applications need.

Libraries are functions that may be useful to many programmers so they are created as reusable code that can be shared. The standard libraries are designed to be very common functions that programmers typically need. This way the programming language is immediately useful to a wider range of programmers. The libraries can be updated and extended without changing the core features of the language itself.

In addition to what the other answers have already said, putting standard functions into a library is separation of concerns:

• It's the compiler's job to parse the language and generate code for it. It's not the compiler's job to contain anything that can already be written in that language and provided as a library.




• It's the standard libraries (the one that's always implicitly available) job to provide core functionality that's needed by virtually all programs. It's not the standard libraries job to contain all the functions that might be useful.




• It's the job of optional standard libraries to provide auxiliary functionality that many programs can do without, but which are still quite basic and also essential for many applications to warrant shipping with standard environments. It's not the job of those optional libraries to contain all reusable code that's ever been written.




• It's the job of user libraries to provide collections of useful reusable functions. It's not the job of user libraries to contain all the code that's ever been written.




• It's the job of an application's source code to provide the remaining bits of code that's really only relevant to that one application.

If you want a one-size-fits-all software, you get something insanely complex. You need to modularize to get the complexity down to manageable levels. And you need to modularize to allow partial implementations:

• The threading library is worthless on the single-core embedded controller. Allowing the language implementation for this embedded controller to just not include the pthread library is just the right thing to do.




• The math library is worthless on the micro-controller that doesn't even have an FPU. Again, not being forced to provide functions like sin() makes life a whole lot easier for the implementators of your language for that micro-controller.




• Even the core standard library is worthless when you are programming a kernel. You cannot implement write() without a syscall into the kernel, and you cannot implement printf() without write(). As a kernel programmer, it's your job to provide the write() syscall, you cannot just expect it to be there.

A language that does not allow for such omissions from the standard libraries is simply no suited for many tasks. If you want your language to be flexibly usable in uncommon environments, it must be flexible in what standard libraries are included. The more your language relies on standard libraries, the more assumptions it makes on its execution environment, and thus restricts its use to environment that provide these prerequisites.

Of course, high level languages like python and java can make a lot of assumptions on their environment. And they tend to include many, many things into their standard libraries. Lower level languages like C provide much less in their standard libraries, and keep the core standard library much smaller. That's why you find a working C compiler for virtually any architecture, but may not be able to run any python scripts on it.

One big reason compilers and standard libraries are separate are because they serve two different purposes (even if they're both defined by the same language spec): the compiler translates higher-level code into machine instructions, and the standard library provides pre-tested implementations of commonly-needed functionality. Compiler writers value modularity just like other software developers do. In fact, some of the early C compilers further split the compiler into separate programs for pre-processing, compiling, and linking.

This modularity gives you a bunch of advantages:

• It minimizes the amount of work needed when supporting a new hardware platform, since most of the standard library code is hardware-agnostic can be re-used.


• A standard library implementation can be optimized in different ways (for speed, for space, for resource usage, etc). Many early computing systems only had one compiler available, and having a separate standard library meant developers could swap implementations to suit their needs.


• The standard library functionality doesn't even have to exist. When writing bare-metal C code for instance, you have a full-featured compiler but most of the standard library functionality isn't there and some things like file I/O aren't even possible. If the compiler was required to implement this functionality, then you couldn't have a standards-conforming C compiler on some of the platforms where you need it the most.


• On early systems, compilers were frequently developed by the company that designed the hardware. Standard libraries were frequently provided by the OS vendor, since they often required access to functionality (like system calls) specific to that software platform. It was impractical for a compiler writer to have to support all of the different combinations of hardware and software (there used to be a whole lot more variety in both hardware architecture and software platform).


• In high-level languages, a standard library can be implemented as a dynamically-loaded library. One standard library implementation can then be used by multiple compilers and/or programming languages.

Historically speaking (at least from C's perspective), the original, pre-standardization versions of the language didn't have a standard library at all. OS vendors and third parties would often provide libraries full of commonly-used functionality, but different implementations included different things and they were largely incompatible with each other. When C was standardized, they defined a "standard library" in an attempt to harmonize these disparate implementations and improve portability. The C standard library developed separate from the language, like the Boost libraries have for C++, but were later integrated into the language spec.

Additional corner-case answer: Intellectual property management

Notable example is implementation of Math.Pow(double, double) in .NET Framework which was purchased by Microsoft from Intel and remains undisclosed even if the framework went open-source. (To be precise, in the above case it is an internal call rather than a library but the idea holds.) A library separated from the language itself (theoretically also a subset of standard libraries) can give the language backers more flexibility in drawing the line between what to keep transparent and what has to remain undisclosed (due to their contracts with 3rd parties or other IP-related reasons).

This is meant as an addition to the existing answers (and is too long for a comment).

There are at least two other reasons for a standard library:

Barrier to Entry

If a particular language feature is in a library function and I want to know how it works, I can just read the source for that function. If I want to submit a bug report/patch/pull request, it's not generally too difficult to code a fix and test case(s). If it's in the compiler, I have to be able to dig into the internals. Even if it's in the same language (and it should be, any self-respecting compiler should be self-hosted) compiler code is nothing like application code. It may take forever to even find the correct files.

You're cutting yourself off from a lot of potential contributors if you go that route.

Hot code loading

Many languages offer this feature to one degree or another, but it would be enormously complicated to hot reload the code that's doing the hot reloading. If the SL is separate from the runtime it can be reloaded.