Skip to content

Commit

Permalink
Plug a hole in package object invalidation
Browse files Browse the repository at this point in the history
scalac isn't sufficiently lazy in the way it deals with package
objects, which causes problems when:

  - a package object has an an ancestor defined under the same
    package
  - the source file defining that ancestor is recompiled but
    the package.scala file is not passed in for recompilation.

This can manifest as a compilation failure with the dreaded
"Symbol $ancestor is missing from the classpath" error
when the class file of the ancestor is _not_ on the classpath, or
undefined results when the stale classfile is included.

Zinc has logic to invalidate package.scala when any of its ancestors
are invalidated. But there is a hole in this logic: if instead an
inner class _within_ one of the ancestors is invalidated, the package
object is not invalidated, despite the fact that Zinc will delete
the class file of and recompile the ancestor.

This commit adds a test that failed with:

```
[info] About to run tests:
[info]  * source-dependencies/package-object-name-inner
...
[info] [debug] Initial set of included nodes: b.O
[info] [debug] Invalidated by transitive inheritance dependency: Set(b.O)
[info] [debug] The following modified names cause invalidation of b.A.Inner: Set(UsedName(o,[Default]))
[info] [debug] Change NamesChange(b.O,ModifiedNames(changes = UsedName(o,[Default]))) invalidates 2 classes due to The b.O has the following regular definitions changed:
[info] [debug] 	UsedName(o,[Default]).
[info] [debug]   > by transitive inheritance: Set(b.O)
[info] [debug]   >
[info] [debug]   > by member reference: Set(b.A.Inner)
[info] [debug]
[info] [debug] New invalidations:
[info] [debug] 	b.A.Inner
[info] [debug] Initial set of included nodes: b.A.Inner
[info] [debug] Previously invalidated, but (transitively) depend on new invalidations:
[info] [debug] Final step, transitive dependencies:
[info] [debug] 	Set(b.A.Inner)
[info] [debug] Invalidated classes: b.A.Inner
[info] [debug] Scala compilation took 0.256750554 s
[info] [debug] done compiling Scala sources
[info] [debug] merging /var/folders/22/g1sv634d11j1d_lqlnhz9p2r0000gn/T/sbt_af9f2f6e/target/early/output-5bdd0cb8-f4f2-4a4d-b629-ee22a2cbba25.jar into /var/folders/22/g1sv634d11j1d_lqlnhz9p2r0000gn/T/sbt_af9f2f6e/target/early/output.ja
[info] [debug] Initial set of included nodes: b.A.Inner
[info] [debug] compilation cycle 2
[info] [info] compiling 1 Scala source to /var/folders/22/g1sv634d11j1d_lqlnhz9p2r0000gn/T/sbt_af9f2f6e/target/classes ...
[info] [debug] Returning already retrieved and compiled bridge: /var/folders/22/g1sv634d11j1d_lqlnhz9p2r0000gn/T/sbt_1e020742/target-bridge-2.12.14.jar.
[info] [debug] [zinc] Running cached compiler 680859f9 for Scala compiler version 2.12.14
[info] [debug] [zinc] The Scala compiler is invoked with:
[info] [debug] 	-Ypickle-java
[info] [debug] 	-Ypickle-write
[info] [debug] 	/var/folders/22/g1sv634d11j1d_lqlnhz9p2r0000gn/T/sbt_af9f2f6e/target/early/output-5bdd0cb8-f4f2-4a4d-b629-ee22a2cbba25.jar
[info] [debug] 	-bootclasspath
[info] [debug] 	/Users/jz/.jabba/jdk/[email protected]/Contents/Home/jre/lib/resources.jar:/Users/jz/.jabba/jdk/[email protected]/Contents/Home/jre/lib/rt.jar:/Users/jz/.jabba/jdk/[email protected]/Contents/Home/jre/lib/sunrsasign.jar:/Users/jz/.jabba/jdk/[email protected]/Contents/Home/jre/lib/jsse.jar:/Users/jz/.jabba/jdk/[email protected]/Contents/Home/jre/lib/jce.jar:/Users/jz/.jabba/jdk/[email protected]/Contents/Home/jre/lib/charsets.jar:/Users/jz/.jabba/jdk/[email protected]/Contents/Home/jre/lib/jfr.jar:/Users/jz/.jabba/jdk/[email protected]/Contents/Home/jre/classes:/Users/jz/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.12.14/scala-library-2.12.14.jar
[info] [debug] 	-classpath
[info] [debug] 	/Users/jz/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.12.14/scala-compiler-2.12.14.jar:/Users/jz/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.12.14/scala-reflect-2.12.14.jar:/Users/jz/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-xml_2.12/1.0.6/scala-xml_2.12-1.0.6.jar:/Users/jz/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/jline/jline/2.14.6/jline-2.14.6.jar:/Users/jz/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/fusesource/jansi/jansi/1.12/jansi-1.12.jar:/var/folders/22/g1sv634d11j1d_lqlnhz9p2r0000gn/T/sbt_af9f2f6e/target/classes
[info] [error] Symbol 'type b.A' is missing from the classpath.
[info] [error] This symbol is required by 'package b.package'.
[info] [error] Make sure that type A is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
[info] [error] A full rebuild may help if 'package.class' was compiled against an incompatible version of b.
[info] [error] one error found
[info] [debug] Compilation failed (CompilerInterface)
[info] [error] x source-dependencies/package-object-name-inner
[info] [error] sbt.internal.inc.BatchScriptRunner$PreciseScriptedError: {line 3} Command failed: 'compile'
[info] [error] Caused by: Compilation failed
[info] [error] 	at sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:335)
[info] [error] 	at sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:420)
[info] [error] 	at sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)
[info] [error] 	at sbt.internal.inc.ProjectStructure.doCompile(IncHandler.scala:678)
[info] [error] 	at sbt.internal.inc.ProjectStructure.$anonfun$startCompilation$12(IncHandler.scala:575)
[info] [error] 	at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:75)
[info] [error] 	at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3313)
[info] [error] 	at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:87)
[info] [error] 	at scala.concurrent.package$.blocking(package.scala:146)
[info] [error] 	at sbt.internal.inc.ProjectStructure.$anonfun$startCompilation$11(IncHandler.scala:573)
[info] [error] 	at scala.util.Success.$anonfun$map$1(Try.scala:255)
[info] [error] 	at scala.util.Success.map(Try.scala:213)
[info] [error] 	at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
[info] [error] 	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
[info] [error] 	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
[info] [error] 	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
[info] [error] 	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
[info] [error] 	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
[info] [error] 	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
[info] [error] 	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
[info] [error] 	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
[info] [info] Log files of all scripted tests run: /var/folders/22/g1sv634d11j1d_lqlnhz9p2r0000gn/T/scripted-logs-b824d2a8
[error] Exception in thread "main" sbt.internal.inc.ScriptedRunnerImpl$ScriptedFailure$1$: 1 (out of 1) scripted tests failed:
[error] 	source-dependencies/package-object-name-inner
```

This is fixed by searching for descendant package objects of any class
defined in the set of invalidated files, rather than just descendants
of invalidated classes.
  • Loading branch information
retronym committed Jun 3, 2021
1 parent d56686b commit 7eeca24
Show file tree
Hide file tree
Showing 6 changed files with 30 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,16 @@ private[inc] class IncrementalNameHashingCommon(
apis: APIs
): Set[String] = {
val findSubclasses = relations.inheritance.internal.reverse _
val invalidatedClassesAndCodefinedClasses = for {
cls <- invalidatedClasses.iterator
file <- relations.definesClass(cls).iterator
cls1 <- relations.classNames(file)
} yield cls1

debug("Invalidate package objects by inheritance only...")
val invalidatedPackageObjects =
transitiveDeps(invalidatedClasses, log)(findSubclasses).filter(_.endsWith(".package"))
transitiveDeps(invalidatedClassesAndCodefinedClasses.toSet, log)(findSubclasses)
.filter(_.endsWith(".package"))
debug(s"Package object invalidations: ${invalidatedPackageObjects.mkString(", ")}")
invalidatedPackageObjects
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
package b

class A {
class Inner {
O.o
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
package b

object O {
def o = ""
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
package object b extends A
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
package b

object O {
def o = 42
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
> compile
$ copy-file changes/O1.scala O.scala
> compile
> checkIterations 3

0 comments on commit 7eeca24

Please sign in to comment.