Check out this cool tool for visualizing packwerk graphs using D3!

This message was imported from the Ruby/Rails Modularity Slack server. Find more info in the import thread.

Hey team! If you’re using packwerk, I wanted to share this fun and experimental tool my colleague created as part of a hackathon that we wanted to share. It generates a packwerk graph using D3:
https://github.com/rubyatscale/visualize_packwerk/tree/main/d3_graph_generator#readme

Here’s an example of what it looks like:

The vision is that it can be even more interactive using filters on the left, so you can select packs to include/exclude based on name, properties of packages, properties of the graph (ancestors/descendants), and more!

If anyone is interested in contributing to this please do :slightly_smiling_face:

Message originally sent by slack user U7195VBSJK9

We actually built a similar internal dashboard using React force graph when we first adopted Packwerk—would love to see if Packwerk itself could just export a dependency graph artifact for generating these kinds of visualizations

Message originally sent by slack user U7195VBSJK9

I wonder how many teams have their own bespoke internal dashboard for looking at dependencies, metadata, deprecated references, etc. :thinking_face:

That sounds cool, although I don’t see packwerk itself wanting to manage that complexity, but it does seem like an awesome candidate for a community maintained solution!

We also built https://github.com/rubyatscale/modularization_statistics for tracking packwerk and related statistics and sending them to datadog

Although if we’re just talking exporting a dependency graph, it does have some APIs to get package and dependency information. We ended up just using parse_packwerk (link) for that since it is a simpler API. I actually had some Graph representations in there for a while but have since removed them.

Would you share any screenshots of your React force graph? Would love to see :raised_hands:

Message originally sent by slack user U7195VBSJK9

I think there’s too much private information unfortunately, but the entire internal dashboard is based on the stuff Shopify shared about Wedge in this blog post

It would be great if we can combine forces on some of this work!

Message originally sent by slack user U70DMI6W1TM

I’ve got a visualization tool built on top of cytoscape.js, which gives you some interactivity for free (zoom, pan, drag nodes, etc.). Here’s the base view of constant references (each node is a constant, the circles are constants defined in a given pack):

Message originally sent by slack user U70DMI6W1TM

and here’s what it looks like when you layer a community detection algorithm on top (the big circles are autodetected communities based on which constants reference each other the most, and the colors are the packs to which the constants currently belong):

Wow! I feel like I’m looking through at a biological system under a microscope!

That’s amazing stuff. I’m excited for our community to collaborate on open-source, interactive software graph visualization tooling. If you wanted to upstream that to visualize_packwerk or another repo, please do :pray: I believe the community has plenty of room for different solutions to experiment with different approaches :bow:

Message originally sent by slack user U70TIGAX94P

This is all fantastic.

I was actually looking to do some analysis on graphs like this (community detection is one avenue I was thinking of). Is the underlying graph data something you‘d be willing to share?

Message originally sent by slack user U70TIGAX94P

I have the feeling that we‘d want to layer something like change locality on top (which files change together), BTW.

Message originally sent by slack user U70TIGAX94P

I‘ve read a paper lately that does some dependency graph analysis

https://pubsonline.informs.org/doi/10.1287/mnsc.1060.0552

It‘s all pretty fancy, but I’m missing the connection to what matters day to day, which is how people work with the code. Layering change locality data somehow on top of dependency data could solve that.

Message originally sent by slack user U70TIGAX94P

For context - I may try to do some academic research on this myself in the near future and I will need data :smile:

It‘d be great to have some robust definitions of the relevant properties of the graph, and maybe come up with… dare I say… metrics