GraphFrames

GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs. It provides
high-level APIs in Scala, Java, and Python. It aims to provide both the functionality of GraphX
and extended functionality taking advantage of Spark DataFrames. This extended functionality
includes motif finding, DataFrame-based serialization, and highly expressive graph queries.
What are GraphFrames? GraphX is to RDDs as GraphFrames are to DataFrames.
GraphFrames represent graphs: vertices (e.g., users) and edges (e.g., relationships between
users). If you are familiar with GraphX, then GraphFrames will be easy to learn. The key differ-
ence is that GraphFrames are based upon Spark DataFrames, rather than RDDs.
GraphFrames also provide powerful tools for running queries and standard graph algorithms.
With GraphFrames, you can easily search for patterns within graphs, find important vertices, and
more. Refer to the User Guide for a full list of queries and algorithms.
creating nodes using pagerank algorithm

# Create a Vertex DataFrame with unique ID column “id”
v = sqlContext.createDataFrame([
(“a”, “Alice”, 34),
(“b”, “Bob”, 36),
(“c”, “Charlie”, 30),
], [“id”, “name”, “age”])
# Create an Edge DataFrame with “src” and “dst” columns
e = sqlContext.createDataFrame([
(“a”, “b”, “friend”),
(“b”, “c”, “follow”),
(“c”, “b”, “follow”),
], [“src”, “dst”, “relationship”])
# Create a GraphFrame
from graphframes import *
g = GraphFrame(v, e)
# Query: Get in-degree of each vertex.
g.inDegrees.show()
# Query: Count the number of “follow” connections in the graph.
g.edges.filter(“relationship = ’follow’”).count()
# Run PageRank algorithm, and show results.
results = g.pageRank(resetProbability=0.01, maxIter=20)
results.vertices.select(“id”, “pagerank”).show()

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s